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Education  and  Training 


FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI) ,  working  in  cooperation  with  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC)  and  with  TRADOC' s  schools, 
conducts  research  to  develop  ways  to  achieve  more  cost-effective 
training  for  the  Army.  In  1987  ARI  joined  with  the  Quartermaster 
School  (QMS)  at  Fort  Lee  to  form  a  partnership  dedicated  to  iden¬ 
tifying  and  solving  Enlisted  Supply  Department  (ESD)  training 
problems.  The  partnership  was  defined  by  a  Letter  of  Agreement 
(LOA)  entitled  "Establishment  of  a  Joint  Training  Technology 
Transfer  Activity  (TTTA)." 

This  report  is  one  result  of  that  partnership.  The  work  was 
carried  out  by  members  of  the  Automated  Instructional  Systems 
Technical  Area  (formerly  the  Logistics  Training  Technologies 
Technical  Area)  of  ARI's  Training  Research  Laboratory  to  supply 
ESD  with  a  method  for  producing  job-relevant  performance 
measures. 

This  and  other  products  of  the  TTTA  were  briefed  to  LTG  Leon 
E.  Salomon,  Commandant,  Logistic  Center  and  Fort  Lee,  and  BG  Paul 
J.  Vanderploog,  Commandant,  Quartermaster  School,  in  the  fall  of 
1989.  It  will  be  used  to  assess  the  impact  of  new  training  de¬ 
velopments  at  QMS.  It  will  also  provide  a  model  for  test  devel¬ 
opment  at  other  TRADOC  schools. 
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THE  DEVELOPMENT  OF  CONTENT-VALID  PERFORMANCE  MEASURES  FOR  THE  76C 
COURSE  AND  FIELD  ASSESSMENT 


EXECUTIVE  SUMMARY 


Requirement: 

To  develop  a  job-relevant  test  battery  to  assess  76C  [pres¬ 
cribed  load  list  (PLL)  clerk  duty  position]  skill  level  in  the 
Advanced  Individual  Training  (AIT)  course  and  in  units.  The 
battery  should  provide  a  standard  to  measure  changes  in  perfor¬ 
mance  resulting  from  new  training  procedures  at  the  U.S.  Army 
Quartermaster  School  (QMS) . 


Procedure: 

We  constructed  a  prototype  performance-oriented  test  bat¬ 
tery,  using  standardized  procedures  for  content-valid  test 
development.  In  doing  so,  we  also  produced  a  model  for  develop¬ 
ment  of  other  content-valid  tests. 

The  prototype  battery  includes  a  sample  of  multiple-choice 
items  from  the  tasks  in  the  PLL  duty  position  of  76C,  and  work- 
sample  tests  from  this  duty  position.  We  administered  the  bat¬ 
tery  at  Forts  Stewart,  Riley,  and  Lewis  to  142  76C  soldiers.  We 
also  collected  supervisor  ratings  and  biographical  information  to 
explore  the  usefulness  of  nontest  data  to  evaluate  soldier  job 
proficiency. 

We  analyzed  test-item  data  and  internal  test  consistency  to 
assess  quality  of  the  battery.  We  also  correlated  supervisor 
ratings  and  biographical  information  with  test  scores.  Finally, 
we  compiled  test  battery  scores  at  the  three  U.S.  Army  Forces 
Command  (FORSCOM)  srces  to  get  baseline  information  on  current 
PLL  skill  levels. 


Findings: 

The  prototype  battery  is  reliable  but  cculd  be  improved 
substantially  by  increasing  the  length  of  the  multiple-choice 
portion.  The  battery  c?n  be  used  to  assess  the  impact  of  new 
training  methods  without  heavy  administrative  demands. 
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Neither  supervisor  ratings  nor  biographical  information, 
e.g.,  job  experience,  correlated  with  test  scores.  The  super¬ 
visor  results  highlight  an  organizational  problem,  i.e.,  76Cs  are 
supervised  by  noncommissioned  officers  (NCOS)  who  may  not  be  in  a 
position  to  accurately  evaluate  76C  performance.  The  biographi¬ 
cal  data  indicate  that  amount  of  job  experience  is  not  necessar¬ 
ily  a  good  indicator  of  skill. 

Test  score  averages  indicated  that  the  76C  skill  level  is 
moderate  (71%  for  the  multiple  choice;  76%  for  the  in-basket) . 

But  a  subtest  of  task  prioritization  showed  poor  performance 
(38%) .  The  first  result  should  not  cause  concern  since  it  mostly 
reflects  manual  form  filling  performance  of  manual  PLL  operations 
being  overtaken  by  computer-aided  procedures.  The  second  result 
reflects  a  higher  order,  nonmechanical  skill,  and  may  need  fur¬ 
ther  examination.  Task  prioritization  is  the  kind  of  skill  being 
addressed  in  another  Training  Technology  Transfer  Activity  (TTTA) 
project — the  Transition  Module  for  76C  MOS. 


Utilization  of  Findings: 

The  test  battery  can  be  used  to  assess  the  impact  of  changes 
in  training  in  the  76C  AIT  course  that  have  been  introduced  as 
part  of  the  TTTA  program.  The  guidance  provided  in  this  report 
for  constructing  content-valid  tests  should  also  be  useful  to 
test  developers  at  QMS  and  other  U.S.  Army  Training  and  Doctrine 
Command  (TRADOC)  schools. 
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THE  DEVELOPMENT  OF  CONTENT-VALID  PERFORMANCE  MEASURES 
FOR  THE  76C  COURSE  AND  FIELD  ASSESSMENT 


INTRODUCTION 


I .  Project  Background 

As  part  of  che  on-going  Training  Technology  Transfer 
Activity  at  the  Quartermaster  School  (QMS) ,  Fort  Lee,  new 
training  methods  and  technologies  are  being  applied  to  the  76C 
Advanced  Individual  Training  (AIT)  course  and  to  on-the-job 
training  in  the  76C  Military  Occupational  Speciality  (MOS) .  In 
this  situation,  the  accurate  assessment  of  the  impact  of  these 
course  modifications  on  trainee  performance  in  the  school  and  on 
the  job  is  essential.  Thus,  performance  measures  must  be 
developed  that  can  provide  a  valid  index  of  trainee  skills  in 
order  to  meas  >*e  the  effects  of  such  course  changes. 

The  development  of  performance  measures  for  the  76C  AIT 
should  be  conducted  with  several  objectives  in  mind.  First,  the 
performance  measures  must  contain  an  accurate  and  representative 
sample  of  tasks  performed  on  the  job.  Second,  they  should 
reliably  measure  the  trainee's  ability  to  perform  those  critical 
tasks.  Third,  an  evaluation  should  be  conducted  on  whether 
additional  non-task-specific  elements  of  the  job  are  critical  to 
successful  performance  and  if  so,  how  the  performance  measures 
will  reliably  assess  trainee  knowledge  of  these  elements. 

Fourth,  the  performance  measures  need  to  be  scoreable  in  a 
consistent,  objective  fashion.  This  is  particularly  the  case  if 
these  tests  are  to  be  placed  within  a  computer-managed 
instructional  (CMI)  system  that  would  provide  automatic  scoring. 
Finally,  scores  should  be  able  to  predict  on-the-job  performance. 

II .  Supply  System  Performance  and  AIT 

The  development  of  high  quality  76C  performance  tests  is  an 
important  element  in  reducing  what  has  become  known  as  the 
transition  problem.  As  Keesee  et  al.  (1980)  among  others  have 
noted,  the  transition  from  76C  AIT  to  performing  in  the  assigned 
unit  has  proven  difficult  for  76C  trainees.  There  are  several 
apparent  reasons  for  this  transition  problem:  1)  76C  AIT 
graduates  often  work  by  themselves  in  a  motor  pool  and  have  few 
people  close  by  who  are  more  knowledgeable  about  supply 
procedures,  2)  post-specific  procedures  for  ordering  parts  and 
maintaining  prescribed  load  lists  (PLLs)  create  some  divergences 
between  what  the  76C  trainee  has  been  taught  in  school  and  what 
he  must  actually  do  on  the  job,  3)  there  is  a  low  system 
tolerance  for  even  minor  errors  and  omissions  in  parts  orders. 
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Valid  performance  tests  based  on  the  findings  of  extensive 
front  end  analyses  can  help  reduce  the  difficulty  of  transition 
by  accurately  assessing  deficiencies  in  trainee  knowledge  and 
skills  which  are  essential  to  successful  job  performance.  In 
addition,  they  guide  the  development  of  appropriate  instructional 
materials  so  that  trainees  can  meet  required  standards  of 
performance  on  the  tests. 

Ill .  Objectives 

A.  Primary.  This  report  will  describe  the  test  development 
methodology  to  be  used  in  the  construction  of  annex  tests  for  the 
four  major  duty  positions  covered  in  the  9  week  76C  AIT  course 
and  provide  the  test  taxonomies  which  determine  the  structure  of 
the  exams.  In  addition,  results  from  field  tests  assessing  the 
utility  of  the  PLL  test  battery  created  according  to  the  taxonomy 
will  be  discussed. 

B.  Secondary.  While  performing  field  tests,  the  opportunity 
to  achieve  some  worthwhile  secondary  objectives  presents  itself. 

A  survey  of  the  job  histories  of  the  participating  76C  soldiers 
can  provide  useful  information  about  the  76C  population, 
particularly  since  the  last  such  survey  was  conducted  in  1986. 

The  relation  between  a  soldier's  test  performance  and  the 
supervisor's  perceptions  of  the  soldier's  everyday  job 
performance  was  also  deemed  to  be  worth  investigating.  Finally, 
the  project  provided  a  means  to  collect  baseline  performance  on 
the  proficiency  of  76Cs  to  help  evaluate  the  effectiveness  of 
future  training  developments  at  QMS. 

METHODOLOGY  FOR  76C  TEST  BATTERY  CONSTRUCTION 
I .  General  Approach 

A.  Job  Analysis.  The  initial  step  in  developing  a  test 
battery  is  an  analysis  of  the  entry  level  job  which  trainees  will 
be  expected  to  perform.  Keesee  et  al.  (1980)  note  that  the  76C 
AIT  graduate  is  expected  to  perform  as  a  journeyman  because 
his/her  supervisor  is  not  in  the  same  MOS,  e.g.,  a  76Y  (Unit 
Supply  Specialist)  or  63B  (Light  Vehicle  Mechanic)  and  not 
necessarily  at  a  higher  skill  level.  While  this  makes  it  more 
difficult  for  the  trainee,  it  obviates  the  necessity  for 
distinguishing  between  the  tasks  performed  by  76C  clerks  with 
different  levels  of  experience  since  they  are  largely  responsible 
for  all  76C  tasks. 

The  Supply  and  Service  (76)  Career  Management  Field  which 
includes  the  76C  MOS  has  been  the  focus  of  a  number  of  recent 
job  and  task  analyses  (Hughes  Aircraft,  1981;  Duncan,  1984; 
TRADOC,  1984).  Because  of  the  existence  of  these  reports,  it  was 
deemed  unnecessary  to  conduct  a  separate  job  analysis  for  the 
current  project.  However,  interviews  were  conducted  with  several 
PLL/TAMMS  (Prescribed  Load  List/The  Army  Maintenance  Management 
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System)  clerks  at  Fort  Lee  and  at  Fort  Carson  as  well  as  with 
instructors  in  the  76C  AIT  program  at  the  QMS  in  order  to  clarify 
certain  aspects  of  the  task  environment. 

The  most  useful  job  analysis  for  the  current  purpose  is  the 
job  analysis  described  in  Duncan  (1984)  which  was  conducted  by 
RCA  Services  and  Educational  Testing  Service  as  part  of  the 
Basic  Skills  Education  Project  (BSEP) .  Critical  tasks  were 
analyzed  in  Skill  Level  10  and  20  for  the  94  highest  density 
MOSs  including  76C.  The  objective  was  to  identify  the 
prerequisite  competencies  or  skills  needed  to  successfully 
perform  the  critical  tasks  in  each  MOS.  Task  lists  were 
developed  on  the  basis  of  existing  materials  such  as  Soldiers 
Manuals  and  lists  developed  by  proponent  schools.  In 
conjunction  with  Subject  Matter  Experts  (SMEs) ,  the  job  analysts 
identified  a)  the  specific  job  or  performance  steps  required  to 
perform  each  task,  b)  the  equipment  required  to  perform  each 
task,  c)  the  knowledge  and  skills  necessary  to  perform  each  task 
and  the  standards  to  which  the  completed  task  should  conform. 

The  critical  tasks  for  the  76C  MOS  determined  by  the  BSEP 
study  are  listed  in  Appendix  A.  The  component  elements  for  each 
of  the  critical  tasks  are  listed  in  Appendix  B.  The  performance 
elements  are  paired  with  the  code  for  the  prerequisite 
competencies  which  were  determined  to  be  essential  to  successful 
element  performance.  The  list  of  prerequisite  competencies 
and  the  corresponding  codes  are  listed  in  Appendix  C. 

B.  Test  Plan.  Given  the  identification  of  the  critical 
tasks  and  the  skills  required  for  their  performance,  the  next 
step  is  the  development  of  the  test  plan.  The  test  plan  can  be 
considered  as  a  template  for  the  construction  of  alternative 
versions  of  a  particular  performance  measure  so  that  there  is  a 
high  degree  of  consistency  between  the  alternate  versions.  When 
th*»  content  of  the  performance  test  closely  resembles  and  samples 
actual  job  tasks  and  duties  (i.e.  possesses  content  validity) 
the  linkage  between  the  items  used  in  the  tests  and  the 
tasks/duties  of  the  job  does  not  require  the  documentation  that 
would  be  necessary  if  the  items  were  more  abstractly  related  to 
job  performance  (e.g.,  aptitude  tests). 

The  test  plan  specifies  the  kind  and  number  of  items  which 
will  be  used  to  measure  trainees'  knowledge  of  how  to  perform  the 
identified  critical  tasks  and  the  skills  associated  with  their 
successful  performance.  In  the  present  case,  the  BSEP  study 
identified  44  critical  tasks  for  the  76C  MOS  which  cover  the  four 
main  duty  positions  of  the  76C  MOS:  PLL  (manual  and  automated) , 
TAMMS,  Shop  Stock  (manual  and  automated)  and  shop  clerk.  Thus, 
the  number  of  critical  tasks  to  be  covered  in  the  individual  AIT 
course  annex  devoted  to  each  duty  position  is  only  a  part  of  the 
total  of  44  tasks.  Due  to  the  size  of  content  material,  the 
performance  tests  for  each  of  these  course  annexes  can  contain  a 
complete  representation  of  the  relevant  critical  tasks  without 
being  overly  long. 
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The  content  of  the  end-of-annex  tests  must  be  based  on  the 
information  taught  in  the  annex.  However,  to  be  predictively 
useful,  the  test  items  must  also  reflect  the  critical  performance 
requirements  of  the  job  environment.  Basing  the  test  content  on 
the  subject  matter  covered  in  the  annex  does  not  require  comment. 
Taking  into  account  the  performance  requirements  of  the  job 
situation  with  respect  to  these  tasks  is  a  more  difficult  issue, 
however.  It  is  not  possible  or  necessarily  desirable  to  exactly 
reproduce  the  job  context  in  the  training  environment.  Those 
aspects  of  the  job  which  seem  to  present  the  greatest 
difficulties  for  the  new  AIT  graduate  deserve  the  most  attention. 

The  results  of  the  job  analyses  mentioned  previously 
indicated  that  several  aspects  of  the  job  environment  cause 
particular  problems.  The  lack  of  adequate  supervisory  technical 
expertise  means  that  the  76C  must  be  more  proficient  at  finding 
information  in  Technical  Manuals.  The  differences  between  school 
and  post  procedures  make  it  more  important  for  76C,  to  have  a 
better  understanding  of  the  nature  of  their  job  activities.  Many 
of  the  differences  are  understandable  or  can  be  seen  as  minor  if 
the  nature  of  the  activity  is  well  understood.  However,  because 
current  training  emphasizes  that  record  keeping  and  supply 
procedures  are  to  be  performed  in  one  specific  way,  there  is 
interference  when  AIT  graduates  have  to  perform  the  activity  in  a 
slightly  different  way  on  the  job.  In  addition,  the  AIT 
graduates  express  concern  about  the  differences  because  they 
believe  that  the  procedures  taught  in  school  are  the  only  correct 
way  of  performing  these  activities. 

From  these  points,  in  addition  to  the  basic  subject  matter 
of  supply  form  completion,  it  would  seem  important  to  test 
trainee  performance  on  job  comprehension,  ability  to  look  up 
information,  and  the  degree  to  which  minor  variations  in 
procedures  can  be  assimilated  without  undue  disruption.  These 
considerations  in  tandem  with  the  job  procedures  themselves  will 
guide  the  formulation  of  types  of  items  which  can  best  measure 
the  knowledge,  skills  and  abilities  essential  to  successful  76C 
performance. 

C.  Item  Construction.  Once  the  test  plan  has  been 
constructed  and  the  item  types  and  their  relative  weights 
specified,  then  item  construction  and  validation  can  begin.  SMEs 
generate  items  whose  characteristics  meet  those  specified  in  the 
test  plan.  The  number  of  items  generated  is  roughly  3  to  4  times 
greater  than  the  total  number  needed  to  allow  for  constructions 
of  alternative  forms. 

When  enough  items  have  been  developed,  they  should  be  tried 
out  on  a  pilot  basis  with  personnel  currently  filling  76C  duty 
positions  and/or  with  trainees  currently  enrolled  in  AIT.  Point 
biserial  correlations  or  other  appropriate  statistical  analyses 
should  be  performed  on  the  overall  test  to  define  its  difficulty 
level  (the  number  of  test  takers  correctly  answering  the  item) 
and  discriminative  power  (the  number  of  students  in  the  top  50% 
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passing  the  item  compared  to  the  number  of  students  in  the 
bottom  50%  passing  the  item) .  The  test  should  have  items  ranging 
in  difficulty  (e.g.,  10%  to  90%  passing  rates)  which  best 
differentiate  high  and  low  performing  students  at  each  level  of 
difficulty. 

Unlike  aptitude  tests,  achievement  tests,  such  as  the  end  of 
annex  test  considered  here,  cannot  contain  items  solely  on  the 
basis  of  their  psychometric  properties  since  there  are  particular 
competencies  which  must  be  exhibited  by  all  AIT  trainees. 

However,  it  is  highly  useful  to  have  a  substantial  proportion  of 
test  items  on  which  a  range  of  performance  can  be  expected;  this 
can  reveal  the  extent  of  individual  differences  between 
trainees'  comprehension  of  the  subject  matter. 

However,  if  the  emphasis  of  the  test  is  solely  upon  the 
proper  classification  of  the  student  as  either  being  a  master  or 
a  non-master  of  the  subject  matter  tested,  then  a  criterion- 
referenced  test  is  indicated.  Here  test  items  would  be  selected 
to  maximally  discriminate  between  masters  and  non-masters  in  the 
region  of  the  cutoff  (criterion)  score.  The  accurate 
classification  of  test  takers  would  thereby  be  enhanced 
(Hambleton  and  De  Gruijter,  1983). 

D.  Validation.  After  the  pilot  testing  of  items  has  been 
completed  and  sets  of  items  sufficient  to  form  two  alternate 
forms  of  the  test  specified  in  the  test  plan  have  been  selected, 
the  validation  of  the  test  should  be  conducted.  The  objective 
is  to  determine  whether  trainees  who  score  well  on  the  test  will 
also  perform  well  on  the  job  in  situations  which  are  highly 
similar  to  the  anticipated  job. 

There  are  a  number  of  ways  to  establish  the  validity  of  a 
performance  test  depending  on  the  relationship  of  the  test  items 
to  the  tasks  performed  on  the  job.  In  the  current  situation, 
since  the  job  tasks  are  form-based  they  can  be  closely 
approximated  by  convential  types  of  paper  and  computer-based 
tests.  This  type  of  content  validation  is  widely  accepted  as  a 
proper  psychometric  procedure  when  the  steps  outlined  above  are 
followed  carefully  (Shrock  et  al.,  1986). 

E.  Reliability  Analysis.  An  essential  part  of  the 
validation  process  is  the  assessment  of  a  test's  or  scale's 
reliability,  i.e.,  the  degree  to  which  the  test  or  scale  provides 
a  stable  index  of  an  individual's  knowledge,  skills,  or  opinions. 
A  common  way  of  considering  reliability  is  to  view  any  particular 
test  as  composed  of  a  random  sample  of  items  from  a  hypothetical 
domain  of  items.  The  purpose  of  an  item  sample  is  to  estimate 
the  measurement  that  would  be  obtained  if  all  the  items  in  the 
domain  were  given (a  person's  true  score).  The  higher  the 
correlation  of  a  sample  of  items  with  the  true  score,  the  more 
reliable  the  sample  of  items  on  the  test.  This  domain  sampling 
model  provides  precise  estimates  of  reliability  when  the  items 
represent  a  single  factor.  If  the  test  items  used  represent 
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several  or  many  factors,  the  estimates  become  less  precise 
because  the  correlations  among  items  are  more  heterogeneous. 
Measurement  theory  is  also  sensitive  to  sampling  error  which 
makes  larger  sample  sizes  of  soldiers  desirable  (e.g.  200-300) . 
For  the  same  reason,  reliability  estimates  are  more  precise  for 
longer  tests.  It  should  be  noted  though  that  estimates  can 
become  very  precise  even  when  the  number  of  items  is  20  or  30 
(Nunnally,  1978) .  Thus  in  most  measurement  problems  there  is 
little  error  in  the  estimation  of  reliability  that  could  be 
attributed  to  random  error  in  the  selection  of  items.  Reliable 
tests  also  tend  to  have  large  variances  of  total  scores  relative 
to  unreliable  tests  of  the  same  length.  The  most  straightforward 
way  of  assessing  reliability  is  the  test-retest  method  in  which  a 
person  completes  a  test  twice.  The  more  reliable  a  test  is,  the 
more  stable  an  individual's  score  would  be  from  test  to  retest. 
Unfortunately,  the  test-retest  method  has  practical  difficulties 
in  using  it  and  in  general  is  more  suited  to  opinion  scales  than 
to  tests. 

Various  statistical  procedures  have  also  been  developed  to 
estimate  reliability  based  solely  on  the  pattern  of  responses 
made  by  individuals  to  the  test  items.  Large  numbers  of 
subjects  (100-300)  are  typically  needed  to  produce  highly  stable 
estimates  by  these  methods  though.  One  concept  derived  from 
these  approaches  is  the  importance  of  the  correlations  between 
item  scores  (pass  rates)  and  test  scores.  Test  items  which  have 
negative  or  zero  correlation  with  overall  test  scores  usually  are 
poor  items  since  people  getting  the  item  correct  are  getting 
lower  overall  test  scores  than  people  missing  the  item. 

Typically,  this  effect  is  produced  by  ambiguous,  poorly  worded, 
misleading,  or  even  factually  incorrect  items.  A  reliable  test 
at  a  minimum  cannot  have  individual  item  scores  negatively 
correlated  with  overall  test  scores. 

Competency  testing,  as  is  found  in  many  Army  training 
situations,  introduces  an  additional  complication  to  construction 
of  tests  with  high  statistical  reliability  since  certain  skills 
must  be  tested  regardless  of  how  many  soldiers  get  them  right 
(or  wrong) .  This  requirement  to  test  all  identified  critical 
tasks,  usually  results  in  less  variation  in  test  scores  since 
some  questions  nearly  everyone  gets  right.  Unfortunately,  the 
statistical  estimation  of  reliability  is  hampered  by  reductions 
in  test  score  ranges. 

If  the  purpose  of  the  test  is  solely  to  classify  students  as 
either  master  (having  achieved  the  cutoff  score)  or  non-master 
(not  reaching  the  cutoff  score)  of  the  content  area,  then  other 
measures  of  reliability  should  be  used.  These  measures  reflect 
the  consistency  of  mastery  classifications  in  test-retest 
situations.  As  previously  noted,  these  measures  of  reliability 
can  be  increased  by  selecting  highly  discriminative  test  items  in 
the  region  of  the  cutoff  score.  This  item  selection  however  can 
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be  at  the  expense  of  uniformly  testing  the  full  content  area  of 
the  test  and  subsequent  diagnosis  of  student  or  instructional 
weaknesses. 

II .  Development  of  Prototype  PLL  Annex  Test  Battery 

In  order  to  provide  a  concrete  example  of  this  test 
development  process  for  the  Technology  Training  Transfer  Activity 
at  the  QMS,  a  prototype  test  battery  for  the  PLL  Annex  was 
developed  and  evaluated  through  the  combined  efforts  of  the  U.S. 
Army  Research  Institute  (ARI)  and  QMS.  Using  the  findings  of  the 
job  analyses  of  the  76C  MOS  (Duncan,  1984) ,  a  test  plan  was 
developed  for  the  PLL  duty  position  by  SMEs  from  the  QMS  working 
with  ARI  researchers  experienced  in  task  analysis  and  test 
development. 

The  test  plan  specified  the  format  and  scoring  procedures  to 
be  followed  for  this  test  and  any  alternatives  versions  of  the 
PLL  test  battery.  Included  within  this  test  plan  is  a  method  (or 
taxonomy)  for  classifying  or  defining  the  test  items  to  be 
constructed.  This  taxonomy  maps  the  major  job  functions  (tasks) 
required  in  performing  the  job  and  the  major  knowledge  and  skill 
categories  (operations)  needed  to  perform  these  functions.  By 
arranging  the  job  functions  as  the  columns  and  the  operations  as 
the  rows,  a  matrix  is  created  which  defines  the  job  space.  The 
matrix  for  the  PLL  (manual)  duty  position  is  presented  in  Figure 
1.  The  matrix  can  also  be  used  after  the  test  is  completely 
developed.  Following  testing,  item  scores  can  be  plotted  into 
the  approporiate  matrix  cell  thereby  indicating  specific  tasks  or 
skill  deficiencies.  Matrices  for  all  the  76C  duty  positions  are 
presented  in  Appendix  D. 

In  order  to  test  PLL  knowledge  and  skills  fully  in  ways 
which  have  high  correspondence  to  the  tasks  performed,  three  item 
types  were  selected  as  appropriate  for  inclusion  in  the  PLL  test 
battery:  1)  multiple-choice,  2)  rankings  (of  task  priorities) , 
and  3)  fill-in  (of  forms) .  Multiple-choice  items  are  useful  for 
measuring  comprehension  of  rules  and  regulations  and  technical 
knowledge.  Rankings  of  task  priorities  measure  the  degree  to 
which  trainees  know  how  to  organize  and  sequence  on-the-job 
activities.  Fill-in  of  forms  measures  working  knowledge  of 
procedures  for  supply  requests  and  inventory  regulation.  The 
resultant  test  plan  for  the  PLL  test  battery  is  presented  in 
Appendix  E. 

Initial  item  development  was  performed  by  QMS  and  ARI 
project  personnel;  then  the  items  were  pilot  tested  on  PLL 
incumbents  at  Forts  Lee,  Carson,  and  Lewis.  After  the  test 
items  were  reviewed  and  where  necessary  modified,  a  prototype 
test  battery  was  then  assembled  which  formed  the  basis  for  the 
present  study. 
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Figure  1.  Matrix  defining  PLL  test  items. 
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MATERIALS  AND  PROCEDURE 


I.  Materials 


The  primary  materials  used  in  the  present  study  were:  a)  a 
34  item  Multiple-Choice  Test,  b)  an  In-Basket  Test  containing  4 
practical  exercises.  Each  exercise  required  the  ranking  of  task 
actions  in  a  problem  scenario  in  terms  of  priority.  Two  of  these 
exercises  also  required  the  completion  of  the  necessary  PLL 
forms  (6)  and  record  postings  (4)  to  perform  the  tasks. 

In  addition,  supplementary  ways  of  assessing  76C  performance  were 
investigated  using  the  Army  Wide  Rating  Scales  (Form  6A) ,  and  a 
PLL  Job  History  Survey. 

A.  Multiple-Choice  Test  (PLL) .  The  Multiple-Choice  Test 
(PLL)  was  devised  according  to  the  Test  Plan  specifications.  The 
test  contained  34  items  which  tested  knowledge  of  different 
functions  and  actions  comprising  the  PLL  duty  position.  These 
items  had  been  culled  from  a  larger  initial  group  of  items 
developed  by  SMEs  at  the  QMS  and  researchers  at  ARI.  The 
initial  group  of  items  was  then  pilot  tested  on  a  sample  of  60 
PLL  clerks  in  order  to  identify  items  which  were 

ambiguous,  unclear  or  otherwise  poor.  Thirty  seven  items  passed 
this  initial  screen  but  3  of  these  items  had  to  be  dropped 
because  of  doctrinal  changes  in  procedure  for  the  referenced 
tasks. 

Although  a  range  of  item  difficulties  was  sought,  items 
could  not  be  excluded  simply  because  they  had  very  high  pass 
rates  if  their  omission  would  leave  a  gap  in  the  taxonomic 
coverage  of  the  test.  U.S.  Army  Training  and  Doctrine  Command 
(TRADOC)  doctrine  mandates  that  AIT  tests  will  cover  all  of  the 
critical  tasks  and  knowledge  that  the  soldier  is  expected  to 
know  at  the  journeyman  level.  Thus,  the  certification  function 
takes  precedence  over  differentiation  of  performance  ability. 

Each  of  the  34  items  had  a  question  stem  less  than  50  words 
long  with  4  response  alternatives,  one  of  which  was  correct. 

Some  of  the  items  could  reasonably  be  answered  only  by  consulting 
the  Supply  Update,  which  was  available  to  the  soldiers  throughout 
the  test  period. 

B.  In-Basket  Test  fPLL) 

1.  Overview.  While  the  items  of  Multiple-Choice  test 
sampled  discrete  aspects  of  job  performances,  the  In-Basket  Test 
was  developed  in  order  to  provide  an  approximation  of  the  whole 
task  activities  and  duties  that  a  PLL  clerk  might  have  to  perform 
during  a  day  "on  the  job"  at  a  motor  pool.  Four  problem 
scenarios  were  created,  each  of  which  described  a  series  of 
overlapping  events  occurring  in  a  brief  span  of  time.  These 
events  (e.g.  supply  requests,  picking  up  parts,  inventory 
control)  required  responses  or  actions  by  the  PLL  clerk  in  the 
correct  sequence.  To  test  soldiers  knowledge  of  task  priority 
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and  scheduling,  each  scenario  was  followed  by  a  Prioritization 
question,  which  listed  the  different  events  described  in  the 
scenario.  The  test  taker  had  to  rank  each  event  numerically  in 
terms  of  the  order  in  which  it  should  be  handled  by  the  PLL 
clerk.  The  Prioritization  items  (P  Items)  had  4-5  events,  which 
were  assigned  a  numerical  ranking  starting  with  1  for  the  first 
event,  to  be  responded  to  by  the  PLL  clerk.  Of  course,  there  was 
no  necessary  connection  between  the  order  of  the  events  as  they 
were  described  in  the  scenario  and  the  correct  order  in  which 
they  should  have  been  handled  by  the  PLL  clerk. 

The  first  two  problem  scenarios  required  the  test  taker  to 
complete  all  necessary  forms  and  make  all  required  postings  as 
would  be  done  on  the  job  by  the  PLL  clerk  in  performing  the 
appropriate  actions.  The  last  two  scenarios  only  required  the 
test  taker  to  complete  the  P  item  for  that  problem  without  having 
to  complete  any  of  the  forms  which  would  be  used  to  perform  these 
actions. 


2.  Form  Fill-in.  The  Form  Fill-In  sections  of  Scenarios  1 
and  2  consisted  of  a  packet  of  blank  or  partially  completed 
supply  forms  used  in  performing  the  standard  functions  of  the  PLL 
clerk  duty  position.  Only  relevant  forms  were  included.  The 
soldiers  had  to  select  the  appropriate  forms  to  complete  the 
scenario  actions  from  that  packet.  Then  they  had  to  fill-in  the 
necessary  information  to  correctly  order  parts,  perform  status 
requests,  and  post  entries  as  dictated  by  the  scenario  events. 

The  Form  Fill-In  sections  were  devised  according  to  the  Test 
Plan  for  the  PLL  Annex.  As  the  plan  indicated,  soldiers  were 
asked  to  fill  out  those  response  blocks  of  the  forms  which  were 
the  minimum  necessary  and  acceptable  to  meet  Army  supply  system 
requirements  for  the  desired  action.  A  distinction  was  made 
between  critical  and  noncritical  response  blocks,  with 
criticality  defined  as  the  likelihood  either  that  errors  would 
not  be  detected  or  that  such  errors  necessitated  the  return  of 
the  supply  request  to  the  PLL  clerk  for  correction.  Errors  in 
response  blocks  defined  as  noncritical  were  capable  of 
correction  at  points  in  the  supply  system  above  the  PLL  clerk 
without  further  clarification  or  input  from  him/her.  Errors  in 
critical  response  blocks  received  a  deduction  of  two  (2)  points 
while  errors  in  noncritical  response  blocks  were  assigned  a  one 
(1)  point  deduction.  A  maximum  of  5  points  could  be  deducted 
from  any  test  taker's  answers  on  an  individual  form,  including  a 
maximum  of  2  critical  errors  (4  points) . 

In  addition  to  supply  or  status  requests  and  inventory 
checks,  the  Form  Fill-In  section  also  required  the  posting  of 
information.  All  errors  in  posting  received  a  one  point 
deduction  for  each  such  error  up  to  a  maximum  of  2  points  for  a 
single  posting  form  (e.g.,  DA  Form-3318,  DA  Form-2064).  The 
correct  posting  of  information  that  was  incorrect  on  the  original 
supply  form  was  not  counted  as  a  posting  error  since  points  had 
already  been  deducted  for  the  mistake  on  the  original  form. 
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C.  DA  Pamphlet  710-2-1  Supply  Update.  Soldiers  taking  either 
the  Multiple-Choice  Test  or  the  In-Basket  Test  were  allowed  to 
consult  DA  Pam  710-2-1  (Supply  Update)  while  answering  test 
items.  The  Supply  Update  is  published  quarterly  and  provides 
the  most  current  version  of  supply  regulations  as  they  apply  to 
the  PLL  duty  position  (and  other  supply  positions) .  Appendices 
to  the  Supply  Update  contain  a  variety  of  tables  listing  various 
supply  codes  and  their  definition.  Because  PLL  clerks  consult 
the  Supply  Update  when  performing  their  duties  and  since  at  least 
some  of  the  information  is  not  expected  to  be  memorized,  it  was 
deemed  reasonable  and  fair  to  allow  the  soldiers  to  have  access 
to  such  information  during  testing. 

D.  Armv-Wide  Rating  Scales  (Form  6A1  and  Rating  Guide 

1.  Rating  Scales.  The  Army-Wide  Rating  Scales  (Form  6A) 
are  a  collection  of  13  rating  scales  to  be  used  in  evaluating 
peer  or  subordinate  job  performance  and  conduct.  [Only  12  scales 
were  used  in  the  present  research  since  Scale  13  (Promotion 
Potential)  was  irrelevant  to  the  objectives.]  The  12  scales  are 
as  follows:  A.  Technical  Knowledge/Skill;  B.  Effort;  C. 
Following  Orders/Regulations;  D.  Integrity;  E.  Leadership;  F. 
Maintaining  Equipment;  G. Military  Appearance;  H.  Physical 
Fitness;  I.  Self-Development;  J.  Self-Control;  K.  Rater's 
Confidence  in  Ratings;  L.  Overall  Effectiveness. 

Most  of  these  scales  are  self-explanatory,  but  K.  Rater's 
Confidence  in  Ratings  deserves  some  comment.  In  this  scale,  the 
persons  doing  the  rating  (peers  or  supervisors)  are  asked  to 
indicate  how  confident  they  are  in  making  the  judgments  of  a 
given  ratee  on  the  dimensions  of  performance  and  conduct  measured 
in  the  other  scales.  Presumably,  the  more  opportunity  to  observe 
the  ratee,  the  greater  the  confidence  should  be  in  the  accuracy 
of  the  ratings. 

The  scales  have  7  scale  points  with  3  behaviorally  anchored 
descriptions  of  low,  moderate  and  high  levels  of  the  dimension. 
The  description  of  behaviors  indicative  of  low  levels  applies  to 
scale  points  1  and  2;  of  mid  levels  to  scale  points  3,  4,  and  5; 
and  of  high  levels  to  scale  points  6  and  7. 

2.  Rating  Guide.  The  Rating  Guide  is  a  written 
explanation  of  sound  rating  practice  which  is  delivered  orally  to 
raters  immediately  prior  to  their  beginning  the  Army-Wide  Scales. 
The  explanation  includes  such  topics  as  the  interpretation  of  the 
behavioral  descriptions  of  scale  points,  the  greater  importance 
of  a  person's  typical  performance  as  opposed  to  occasional  highs 
and  lows,  halo  errors,  the  avoidance  of  undue  weight  to  the  most 
recent  events,  and  individual  differences  on  different  dimensions 
of  performance.  In  addition,  the  Rating  Guide  provides  a 
description  of  the  correct  way  to  use  and  fill  out  the  rating 
scales  particularly  when  the  rater  has  two  or  more  ratees  to 
judge. 
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E.  PLL  Job  History  Survey.  The  PLL  Job  History  Survey 
consists  of  9  questions  given  to  all  test  takers  concerning 
their  current  status  in  the  76C  MOS  and  previous  related 
experience  and  training.  The  survey  was  intended  to  provide  a 
rough  picture  of  the  job  related  characteristics  of  the  current 
sample  of  76Cs  participating  in  the  study.  Inasmuch  as  the  last 
limited  survey  occurred  more  than  4  years  ago,  the  survey  should 
provide  a  useful  picture  of  the  personnel  currently  staffing  76C 
duty  positions  and  provide  an  indication  of  any  changes  over  time 
in  their  job  related  training  and  experience.  In  addition,  the 
survey  permits  the  analysis  of  the  effects  of  experience  and 
training  histories  on  soldiers  test  scores.  None  of  the  survey 
questions  involved  personal  conduct  on  or  off  the  job. 

II .  Experimental  Procedure 

A.  Subjects.  A  sample  of  142  male  and  female  enlisted 
soldiers  currently  classified  in  the  76C  MOS  were  given  either 
the  Multiple-Choice  Test  or  the  In-Basket  Test.  The  soldiers 
were  stationed  at  Fort  Bragg,  Fort  Riley,  or  Fort  Lewis,  all  of 
which  were  U.S.  Army  Forces  Command  (FORSCOM)  sites  with  high 
densities  of  currently  active  76Cs.  All  soldiers  were  required 
to  have  had  at  least  2  months  of  experience  as  PLL  clerks 
sometime  in  the  past  2  years  (i.e.,  starting  with  March  1987). 

The  testing  at  each  site  occurred  over  a  period  of  4  consecutive 
days.  The  testing  across  sites  occurred  between  March  1989  and 
June  1989.  In  addition,  a  total  of  66  76C  supervisors  provided 
ratings  of  their  76C  subordinates  using  the  Army-Wide  Rating 
Scales  (Form  6A) .  Only  those  supervisors  who  had  one  or  more  of 
their  subordinates  participating  in  this  study  were  asked  to 
provide  ratings.  With  the  small  exception  of  five  76C  sergeants, 
the  supervisors  were  noncommissioned  officers  (NCOs)  holding 
either  the  63B  MOS  or  76Y  MOS. 

B.  Rating  Scales.  The  supervisors  rated  their  subordinates 
using  11  (of  12)  of  the  previously  described  rating  scales 
originally  developed  to  assess  the  effectiveness  of  first  term 
soldiers  participating  in  the  Army  Selection  and  Classification 
Project.  The  ratings  looked  at  three  aspects  of  the  soldiers: 
performance,  overall  effectiveness,  and  NCO  potential.  However, 
in  the  current  study,  the  NCO  potential  scale  was  not  used. 

To  determine  the  correlation  between  test  scores  of  non- 
supervisory  76C  soldiers  and  supervisory  perceptions  of 
performance,  supervisors  were  asked  to  rate  the  76C  soldiers  on 
certain  criteria.  Depending  on  availability,  first  and  second 
line  supervisors  were  asked  to  rate  all  of  their  subordinates  who 
were  also  participating  in  this  study.  Of  the  96  supervisors 
that  participated,  80%  were  first  line  supervisors  and  20%  were 
second  line.  A  total  of  116  soldiers  were  evaluated  by  either 
their  first  line  or  second  line  supervisor.  Some  supervisors 
were  not  available  so  test  scores  from  the  remaining  subordinates 
could  not  be  correlated  with  supervisory  ratings. 
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The  supervisors  were  asked  to  describe  the  length  and  level 
of  their  supervision  of  the  participating  subordinates.  Specific 
supervision  time  ranged  from  less  that  one  month  to  more  that  12 
months.  The  majority  of  soldiers  had  been  supervised  more  than 
4  months,  22%  fcr  4-6  months,  29%  for  7-12  months,  and  23%  more 
than  12  months.  Only  a  small  percentage  had  been  supervised 
less  than  4  months,  5%  less  than  1  month  and  20%  1-3  months. 

There  was  no  major  discrepancy  in  supervision  time  when  the 
information  was  broken  down  by  sites.  On  the  average  25%  of  the 
soldiers  in  each  group  had  been  supervised  less  than  4  months. 

C.  Experimental  Design  and  Procedure 

1.  Overview.  Soldiers  were  randomly  assigned  to  either 
the  Multiple-Choice  Test  group  or  the  In-Basket  Test  Group.  All 
soldiers  completed  the  Job  History  Survey  before  taking  their 
assigned  test.  All  supervisors  received  rater  training  prior  to 
completing  the  Army-Wide  Rating  Scales  (Form  6A) . 

The  procedure  used  to  administer  the  performance  testing 
and  supervisory  ratings  was  consistent  across  different  sessions 
and  test  sites.  Soldiers  and  supervisors  were  notified  by  their 
commanding  officer  (CO)  in  the  week  prior  to  test  administration 
to  report  to  a  designated  classroom  on  post  and  participate  in  a 
survey  and  experimental  project  sponsored  by  the  QMS  and  Army 
Research  Institute.  Two  sessions  of  2  hours  each  were  scheduled 
for  each  day  of  the  week  of  testing  allotted  by  FORSCOM  for  that 
site.  Personnel  were  scheduled  for  a  particular  session  based  on 
their  assigned  unit.  Thus,  in  any  given  session,  soldiers  were 
mostly  from  one  or  two  companies.  Supervisors  were  more 
irregularly  distributed  throughout  the  sessions  for  the  week. 
Thus,  supervisors  did  not  necessarily  attend  the  same  session  as 
their  subordinates. 

2.  Location  of  Testing.  The  sessions  took  place  in 
moderately  large  classrooms  and  meeting  rooms  which  contained 
either  several  large  tables  seating  4  to  6  people  each  or  a 
conventional  classroom  with  rows  of  individual  desks.  The  rooms 
were  relatively  free  from  noise  and  other  disturbances.  As 
personnel  entered  the  room,  the  researchers  asked  supervisors  to 
identify  themselves.  Supervisors  were  then  directed  to  sit  at  a 
separate  table  or  section  of  the  room  from  the  76Cs. 

At  each  place  at  the  table  or  desk  designated  for  the  76Cs, 
the  following  materials  were  set:  a)  Multiple-Choice  Test  or  In- 
Basket  Test,  b)  DA  Pam  710-2-1,  c)  one  pencil.  At  places  for 
supervisors,  one  Army-Wide  Rating  Scale  packet  and  a  pencil  were 
set. 


3.  Testing  Instructions.  The  test  session  began  with  the 
chief  on-site  researcher  introducing  the  data  collection  staff 
and  explaining  the  purpose  of  the  study.  The  voluntary  nature  of 
soldiers  participation  was  emphasized  at  this  time.  It  was 
further  explained  that  performance  scores  were  to  be  used  only  to 
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evaluate  training  procedures  at  QMS  and  would  definitely  not 
become  part  of  their  service  record.  All  attending  soldiers 
agreed  to  participate  under  these  terms. 

At  this  point,  the  researchers  split  up,  with  one  assigned 
to  the  supervisors  and  the  other  to  the  76Cs.  The  researcher 
with  the  supervisors  delivered  the  rater  training  as  specified  in 
the  Training  Guide  and  then  monitored  the  completion  of  the 
ratings,  answering  any  procedural  questions.  The  researcher  with 
the  76Cs  informed  them  that  they  would  be  completing  either  a 
multiple-choice  test  or  an  in-basket  type  test  involving  form 
fill-out  on  PLL  procedures. 

All  soldiers  were  told  that  they  had  essentially  all  the 
time  they  wanted  to  complete  the  tests  and  could  make  use  of  the 
materials  from  the  Supply  Update.  However,  they  were  asked  to 
refrain  from  discussing  questions  or  sharing  answers  with  other 
participants.  In  addition,  they  were  asked  to  avoid  revealing 
the  test  items  to  other  personnel  after  they  left  the  test 
session. 

Soldiers  were  asked  to  first  complete  the  Job  History  Survey 
before  starting  with  the  assigned  test.  Soldiers  who  had  the 
Multiple-Choice  Test  were  told  to  circle  the  correct  response 
alternative  to  each  question.  Soldiers  with  the  In-Basket  Test 
were  told  to  read  the  problem  scenarios  and  prioritize  the 
various  actions  that  should  be  taken.  Then,  where  indicated, 
they  were  to  complete  the  forms  and  postings  that  would  be 
required  to  actually  perform  the  specified  actions. 

4.  Time  Allowed.  All  supervisors  and  soldiers  completed 
their  assignments  within  the  two  hour  period  allotted.  The 
Multiple-Choice  Test  was  completed  in  an  average  of  35  minutes 
while  the  In-Basket  Test  took  an  average  of  65  minutes  to 
complete.  Supervisors  performed  the  rating  task  in  about  15 
minutes  plus  approximately  10  minutes  to  go  over  the  rater 
training  materials.  When  a  soldier  had  finished,  their  materials 
were  collected  and  they  were  excused  from  the  session. 

Supervisors  and  soldiers  generally  had  few  questions  about  their 
tasks  and  all  the  sessions  at  the  3  sites  took  place  under 
generally  equivalent  circumstances. 

5.  Scoring. 

a.  Multiple-Choice  Test.  This  test  was  scored  in 
straightforward  fashion,  with  each  item  worth  one  point  and  the 
total  score  being  the  sum  of  the  number  correct. 

b.  In-Basket  Test.  The  In-Basket  Test  was  necessarily 
more  complex  to  score  than  the  Multiple-Choice  test.  The  In- 
Basket  Test  had  two  types  of  items:  prioritizing  (rank 
orderings)  of  4  and  5  actions  and  fill-in  of  (form)  blanks.  The 
rank  orderings  were  scored  one  point  for  each  correct  ranking 
with  a  total  of  4  or  5  points  available  fcr  each  of  the  4 
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prioritizing  items.  There  were  two  form  completion  exercises 
with  each  item  containing  several  forms  to  complete.  Each  form 
had  5-10  blanks  to  fill  in,  but  not  all  ■‘'he  blanks  are  of  equal 
importance  or  criticality  in  the  successful  completion  of  a 
supply  request.  For  this  reason,  mistakes  had  differing  degrees 
of  severity,  reflected  in  the  scoring  system  used.  Mistakes  in  1 
to  2  critical  blocks  on  a  single  form  resulted  in  a  2  point  and  4 
point  deduction  respectively  from  the  points  available  for  that 
item.  Mistakes  in  3  or  more  critical  blocks  on  a  single  form 
resulted  in  the  deduction  of  all  ten  points  available  for  that 
form.  Mistakes  in  noncritical  blocks  resulted  in  one  point 
deductions  from  the  item  score  up  to  a  maximum  of  2  such  points 
deducted. 

In  addition  to  completing  necessary  supply  forms,  soldiers 
also  had  to  post  this  information  when  appropriate.  All  errors 
in  posting  received  a  1  point  deduction  for  each  error  up  to  a 
maximum  of  2  points  for  a  single  posting  form  (e.g.,  DA  Form- 
3318,  DA  Form-2064).  Logically  speaking,  postings  could  contain 
incorrect  information  either  because  the  initial  entries  on  the 
forms  were  incorrect  and  were  transferred  to  the  postings  this 
way  or  because  initial  correct  information  was  transferred 
incorrectly  in  posting.  The  second  case  is  clearly  a  mistake  in 
the  act  of  posting.  The  first  case,  however,  represents  the 
(accurate)  posting  of  incorrect  information.  Since  points  were 
deducted  on  the  original  forms  for  such  errors,  it  was  deemed 
inappropriate  to  double  the  penalty  for  such  errors  simply 
because  they  were  posted  elsewhere,  as  long  as  such  postings  were 
performed  correctly.  Therefore,  no  additional  points  were 
deducted  for  such  postings. 

RESULTS 

I .  Multiple-Choice  Test  (PLL) 

A.  Descriptive  Statistics 

1.  Overall  Test  Scores.  The  Multiple-Choice  Test  (PLL) 
contained  34  items  systematically  covering  the  major  functional 
tasks  involved  in  performing  the  PLL  position.  Combined  results 
across  test  sites  showed  that  scores  ranged  from  20  to  31  with 
24.5  (71%)  being  the  average  score  (n=71,  SD=2.7)  The  modal 
score  was  25.  (Each  item  had  a  value  of  1  point.) 

The  tests  comprising  the  Battery  were  completed  by  the 
soldiers  with  very  few  questions  about  the  interpretation  of 
questions  or  comments  that  the  test  content  was  inappropriate. 

The  one  topic  that  elicited  some  comments  were  the  items  on  the 
Multiple-Choice  Test  which  tested  some  aspect  of  the  DA  Form 
3318.  Many  soldiers  were  currently  using  an  automated  PLL  which 
takes  on  the  record  keeping  function  of  the  DA  Form  3318s. 
Nevertheless,  all  soldiers  agreed  that  they  had  received  training 
on  the  DA  Form  3318s  in  AIT  or  in  the  base  schools  and  that  it 
was  still  required  for  Skill  Qualification  Test  (SQT) . 
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The  scores  did  display  some  differences  as  a  function  of 
site  as  measured  by  the  Kruskal-Wallis  one-way  ANOVA  test. 

Fort  Riley,  (n=23) ,  and  Fort  Bragg  (n=16)  had  similar  averages, 

25  (SD=2 . 1)  and  25.5  (SD=3.5)  respectively.  The  majority  of  the 
scores  fell  between  20  and  26,  while  37%  of  the  scores  were  28  or 
above.  Fort  Lewis  (n=32 ;SD=2 . 2)  had  a  significantly  lower 
average  score  of  23,  (chi-square=13 . 7  df=70)  that  ranged  from  20 
to  30.  Most  of  those  scores  were  between  20  and  25  with  only  6% 
at  28  or  above  (see  Table  1) . 

2.  Automated  PLL  Scores.  The  Multiple-Choice  Test 
contained  items  which  dealt  with  automated  as  well  as  manual  PLL 
procedures.  Because  some  sites  were  using  PLL  automated 
procedures  exclusively,  rather  than  a  mixture  of  manual  and 
automated  procedures,  it  was  decided  to  additionally  analyze  test 
scores  on  the  PLL  automated  items  alone  (Autoscores) .  The 
purpose  was  to  see  if  any  significant  differences  could  be  found 
between  the  overall  test  scores  (Testscores)  and  the  scores  on 
the  Automated  only  items  (all  the  items  except  Items  14,  15,  17, 
18,  20,  29,  and  30).  A  chi-square  test  revealed  no  significant 
differences  between  the  Testscores  and  the  Autoscores  expressed 
as  proportions. 

3.  Reliability.  The  reliability  (shown  in  Table  2)  of  the 
test  was  measured  by  the  Cronbach  alpha  procedure.  The 
reliability  coefficient  for  the  entire  test  was  .44  (n=71) .  The 
removal  of  one  item  (#17)  increased  the  reliability  coefficient 
to  .48  .  The  relatively  low  reliability  could  be  due  to  a  number 
of  factors.  The  test  length  may  have  been  too  short  although 
coverage  of  all  the  major  tasks  was  ensured  by  the  taxonomy  used 
to  guide  test  construction.  In  addition,  because  this  test  could 
be  used  as  part  of  a  certification  process,  a  number  of  items 
have  to  be  included  even  though  the  pass  rates  are  very  high. 

This  could  produce  restriction  of  range  effects  for  the  test 
scores  which  reduce  the  observed  reliability. 

B.  Item  Analyses 

1.  Pass  Rates.  The  test  items  displayed  a  considerable 
range  of  pass  rates.  Five  of  the  items  had  100%  pass  rates  while 
six  other  items  had  pass  rates  between  90%  and  97%.  Conversely, 
three  items  had  pass  rates  below  25%.  The  remaining  items  were 
fairly  evenly  distributed  between  these  extremes. 

2.  Item-Test  Correlations.  An  analysis  was  made  of  the 
correlation  between  soldier's  test  scores  and  pass-fail  scores  on 
the  individual  items  as  a  way  of  screening  for  confusing, 
ambiguous  or  otherwise  poor  items.  A  negative  or  zero 
correlation  between  the  two  indicates  that  the  better  performers 
are  doing  the  same  or  worse  on  that  item  than  the  poorer 
performers,  not  a  desirable  characteristic.  The  items  with  100% 
pass  rates  are  of  course  not  analyzable  with  this  procedure. 
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Table  1 


Mean  Performance  Scores  for  Test  Battery  Components  by  Post 


Post 

Multiple- 
Choice 
Number/ (% 
Correct) 

In-Basket 

Errors/ 

(%  Correct) 

Prioritizing 
Number/ (% 
Correct) 

Supervisor 

Ratines 

Riley 

25.6/  75% 

36.7/  80% 

7.1/  37% 

4.9 

Bragg 

25.5/  75% 

59.5/  67% 

8.1/  42% 

5.1 

Lewis 

23.3/  68% 

41.3/  77% 

5.1/  26% 

4.9 

Table  2 

Reliability  of  Test  Battery  Components 


Test  Component 
Multiple-Choice 
In-Basket  Forms 
Prioritizing 

*Item  17  deleted 


Reliability 

0.48* 

0.65 

0.72 


Note.  The  values  represent  Cronbach's  coefficient  alpha  of  internal 
consistency. 
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Only  one  item  (#17)  had  a  slight  negative  correlation  with  test 
scores  (r=-.01).  Two  other  items  displayed  positive  correlations 
but  close  to  zero.  As  noted  above,  Item  17  was  the  only  item 
whose  removal  significantly  increased  overall  test  reliability. 

II.  In-Basket  Test  (PLL) 

A.  Descriptive  Statistics.  Because  of  the  complex  scoring 
structure  on  the  form  fill-in  items,  results  are  reported  in 
terms  of  the  total  error  points  assessed  on  each  of  the  forms 
rather  than  overall  test  scores.  The  prioritizing  rank  order 
scores  however  are  recorded  as  points  correct. 

1.  Prioritizing  Scores  (P  Scores).  A  total  of  66  76Cs 
completed  the  prioritizing  items  across  the  three  sites.  The 
overall  scores  ranged  from  16  (out  of  a  maximum  of  19)  points  to 
0  correct.  The  mean  score  was  6.1  or  38%  correct  rankings  with  a 
SD  of  3.5.  It  should  be  noted  that  since  the  items  called  for  a 
rank  order,  one  mistake  would  result  in  two  or  more  incorrect 
ranks.  Thus,  the  absolute  scores  overestimate  the  number  of 
errors  committed  by  the  soldiers  although  this  doesn't  affect 
their  relative  position.  The  majority  of  scores  (51.9%)  were 
clustered  between  3  and  9  points  correct  with  a  bimodal 
distribution  at  4  and  7  points.  A  Kruskal-Wallis  one  way  ANOVA 
revealed  a  significant  effect  of  site  on  P  Scores  (chi- 
square=9.65,  p<.05  corrected  for  tie  ranks).  Fort  Lewis  had  the 
lowest  mean  P  Score  (m=5.1,  n=37)  with  Fort  Riley  next  (m=7.5, 
n=22)  and  Fort  Bragg  highest  at  m=8.1  (n=7) .  Table  1  shows 
these  P  scores  by  site. 

2.  Form  Fill-in  Error  Scores  (E  Scores).  A  total  of  74 
76Cs  completed  the  form  fill-in  items  across  the  three  sites. 
Eight  of  these  soldiers  failed  to  complete  the  prioritizing 
items,  accounting  for  the  difference  in  N  between  E  Scores  and  P 
Scores.  As  noted,  the  E  Score  represents  points  deducted  for 
mistakes  in  form  fill-in  (out  of  180  maximum) .  The 

E  Scores  ranged  from  a  best  score  of  -18  to  the  lowest  score  of 
-81.  The  mean  E  Score  was  -42.8  (76%  correct)  with  a  SD  of  20.8. 
Scores  were  evenly  distributed  in  this  range  with  only  a  slight 
hump  between  -38  and  -41.  A  Kruskal-Wallis  one  way  ANOVA  did  not 
reveal  a  significant  effect  of  site  on  E  Scores.  Table  1  shows 
these  E  scores  by  site. 

3.  Reliability.  The  reliability  of  the  In-Basket  Test  was 
assessed  by  Cronbach  alpha.  The  overall  reliability  coefficient 
was  0.65  for  the  10  E  Score  (Forms  and  Posts)  items  (see  Table 

2) .  The  exclusion  of  Form  4  raised  the  reliability  coefficient 
slightly  to  0.67.  Since  the  In-Basket  Test  was  a  power  test 
rather  than  a  speed  test  there  is  some  question  as  to  the 
interpretation  of  missing  values.  As  a  check  on  this, 
reliability  was  analyzed  with  all  missing  values  on  Form  and  Post 
items  reclassified  as  errors.  The  reliability  coefficient  was 
not  greatly  different  in  this  case  however  (0.59).  The 
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reliability  coefficient  of  the  P  Score  (Situation  Prioritizing) 
items  was  0.72  with  missing  values  excluded.  No  item 
significantly  decreased  the  overall  reliability. 

B.  Item  Analyses 

1.  Error  Rates.  Because  of  the  nature  of  the  items  in 
form  fill-in,  errors  were  a  more  meaningful  analysis  than  pass 
rates.  An  item  in  this  part  of  the  In-Basket  Test  represented  a 
separate  form  that  needed  to  be  completed  to  fulfill  the  supply 
action  required  by  the  problem  situation.  The  two  problem 
scenarios  which  specified  form  fill-in  required  a  total  of  six 
action  forms  and  five  posting  forms  (Forms  1  to  6  and  Posts  1  to 
5)  . 


Unlike  the  Multiple-Choice  Test,  the  incidence  of  incomplete 
or  non-completed  items  was  moderate  to  high  although  soldiers 
essentially  had  unlimited  time  to  complete  the  In-Basket  Test. 

Two  contributing  factors  to  this  effect  were  the  greater  amount 
of  time  necessary  to  complete  the  In-Basket  Test  compared  with 
the  Multiple-Choice  Test  (60  minutes  vs  30  minutes)  and  the 
inherent  uncertainty  in  selecting  and  completing  forms  to  problem 
scenarios  which  do  not  specify  the  response  alternatives. 

Missing  value  rates  ranged  from  1.4%  (Form  6)  to  91.9%  (Form  4) 
with  the  average  at  32%,  if  Form  4  is  included  (clearly  a  poor 
item)  or  14.7%,  if  Form  4  is  excluded.  The  Post  items  averaged 
about  23%  missing  values. 

For  completed  Forms,  error  rates  were  lowest  for  Forms  1  and 
6,  in  which  over  85%  of  the  soldiers  had  an  error  score  of  -2  or 
better.  Error  rates  were  roughly  twice  as  high  with  Forms  2,  3, 
and  5  since  only  about  40%  of  soldiers  had  error  scores  of  -2  or 
better  with  these  items.  The  large  number  of  nonresponses  to 
Form  4  allowed  only  6  cases  to  analyze;  however,  those  who  did 
complete  it  had  low  error  rates.  This  suggests  that  the  Form 
itself  was  not  likely  the  reason  for  the  nonresponses,  but  rather 
it  was  probably  the  phrasing  and  interpretation  of  the  problem 
scenario  which  led  soldiers  to  think  the  Form  was  not  required. 

Error  rates  on  completed  Posts  1  to  5  varied  considerably 
as  well.  Posts  4  had  the  lowest  error  rates  with  80%  scoring  -2 
or  better  while  Posts  1  had  the  highest  error  rates  with  more 
than  80%  scoring  worse  than  -2.  Posts  2,  3  and  5  were  roughly 
comparable  with  80%  scoring  -4  or  better. 

2.  Inter- item  Correlations.  An  analysis  was  made  between 
soldiers's  overall  E  scores  and  the  E  Scores  on  individual  items 
as  a  way  of  screening  for  poor,  confusing,  or  ambiguous  items.  A 
negative  or  zero  correlation  indicates  an  inverse  or  no  relation 
between  overall  E  Scores  and  the  E  Scores  on  an  individual  item 
which  is  undesirable.  Only  Form  4  had  a  negative  or  zero  item- 
total  correlation  (-0.03)  while  the  other  items  mostly  had 
correlations  in  the  .30s  and  .40s. 
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A  similar  analysis  was  performed  between  soldiers  overall 
P  scores  and  the  scores  on  individual  items.  All  items  had  a 
positive  item-total  correlation  with  the  majority  of  the 
correlations  falling  between  r=.25  and  r=.40. 

III.  Job  History  Survey 

A.  Descriptive  Statistics.  The  Job  History  Survey  was 
completed  by  all  145  non-supervisory  76C  soldiers  participating 
in  the  study  (see  Appendix  F) .  The  three  participating  sites 
contributed  different  proportions  of  soldiers  to  the  overall 
sample.  Fort  Lewis  had  the  most  representation  with  49% 
followed  by  Fort  Riley  with  32%,  and  then  Fort  Bragg  which  had 
19%.  The  majority  of  the  soldiers  were  Specialist  Four  (E-4) 

(54%) .  The  next  most  frequent  rank  was  Private  First  Class  (E-3) 
with  26%.  The  remainder  were  either  Privates  (E-2)  or  Sergeants 
(E-5) .  The  most  frequent  current  duty  position  of  the  soldiers 
was  PLL  clerk  (63%),  followed  by  TAMMS  clerk  25%  and  shop  clerk 
12%.  Half  of  the  soldiers  had  spent  between  one  to  two  years  in 
their  current  duty  position;  while  34%  had  spent  less  than  one 
year  in  their  position  and  16%  had  spent  three  or  more  years. 

Since  there  are  several  duty  positions  in  the  76C  MOS  it  is 
common  for  soldiers  to  rotate  assignments  between  these  duty 
positions.  For  this  reason  it  is  possible  for  a  76C  currently 
performing  in  the  TAMMS  or  shop  clerk  positions  to  have  had 
previous  PLL  experience.  Therefore,  soldiers  were  asked  to 
indicate  their  duty  position  prior  to  their  current  one,  if  any. 
Nearly  half  of  the  soldiers  had  been  PLL  clerks  in  their 
previous  position,  while  30%  had  been  in  TAMMS,  and  21%  had  been 
shop  clerks.  However,  all  of  the  soldiers  had  at  least  two 
months  of  PLL  experience  sometime  in  the  past  two  years. 

Soldiers  were  also  asked  about  the  nature  of  their  job 
training.  The  overwhelming  majority  (98%)  of  the  soldiers  were 
graduates  of  the  AIT  course  given  at  the  QMS,  Fort  Lee.  The 
remaining  soldiers  had  been  reassigned  from  other  MOS  into  the 
76C  MOS.  At  some  time  after  graduating  from  the  AIT  course,  13% 
of  the  soldiers  had  been  given  refresher  training.  Sixty-eight 
percent  of  the  soldiers  had  attended  a  course  on  PLL  skills  given 
at  some  duty  location.  These  courses  usually  last  one  to  two 
weeks  and  provide  both  review  of  PLL  procedures  and  familiar¬ 
ization  with  the  post-specific  variations  constituting  local 
practice.  Students  have  to  pass  a  final  test  to  graduate  from 
these  courses. 

Soldiers  were  also  asked  to  rate  their  level  of  job 
knowledge  as  well  as  the  use  of  reference  materials  to  assist 
them  in  performing  their  tasks.  Fifty-five  percent  rated 
themselves  as  having  moderate  levels  of  job  skill,  while  43% 
considered  themselves  as  having  high  levels  of  job  knowledge. 

Only  3%  considered  themselves  as  having  low  job  knowledge. 

Most  soldiers  reported  using  reference  materials  occasionally  in 
the  course  of  their  job,  with  lesser  percentages  reporting  either 
frequent  use  or  low  levels  of  use,  27%  and  19%,  respectively. 
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The  data  were  broken  down  by  site  to  see  if  there  were  any 
location  variations  in  biographical  data.  There  were  very  few 
differences  detected  in  the  response  between  the  various  sites 
and  none  of  these  seemed  significant  for  the  interpretation  of 
the  performance  results.  For  example,  at  Fort  Bragg  and  Fort 
Lewis  half  of  the  soldiers'  prior  duty  position  was  as  a  TAMMS 
clerk  while  at  Fort  Riley  two-thirds  were  previously  PLL  clerks. 

B.  Test  Score  Cross-Correlations 


1.  Multiple-Choice  Test  Scores  by  Job  History  Survey 
Variables.  The  overall  scores  on  the  Multiple-Choice  Test  were 
correlated  with  soldiers'  responses  to  the  Job  History  Survey 
items  concerning  their  MOS  related  experiences.  Most  of  the  job 
experience  variables  showed  low  or  no  correlation  with  test 
scores  either  for  all  sites  together  or  by  individual  sites. 

The  variables  showing  nonsignificant  correlations  were:  rank, 
prior  duty  position  held,  length  of  time  spent  in  the  current 
duty  position,  whether  they  had  received  "shadow  school"  training 
on  post,  or  the  degree  to  which  they  used  reference  materials  on 
the  job.  The  soldiers'  current  duty  position  showed  a  marginally 
significant  correlation  with  Testscores  (the  complete  34  item 
test)  (r=.16,  p< . 08 )  and  a  significant  correlation  with 
Autoscores  (r=.20,  pc. 05).  As  would  be  expected,  soldiers 
currently  in  the  PLL  duty  position  had  (slightly)  higher  scores 
than  soldiers  currently  in  the  related  76C  duty  positions  of 
TAMMS  Clerk  and  Shop  Clerk.  On  the  other  hand,  a  surprising 
negative  correlation  was  found  for  soldiers  who  had  been  given 
refresher  training  at  QMS  (r=  -.19,  pc. 05),  although  such 
training  may  be  specified  in  part  based  on  performance 
difficulties.  Interestingly,  the  strongest  correlation  was  found 
between  self-reports  of  degree  of  job  knowledge  and  both 
Testscores  and  Autoscores  (r=.24,  pc. 05) 

2.  In-Basket  Test  Scores  by  Job  History  Survey  Variables. 
The  correlations  between  soldiers'  responses  on  the  Job  History 
Survey  and  their  In-Basket  Test  Scores  showed  a  similar  pattern 
to  that  obtained  with  the  Multiple-Choice  Test  Group  as  we  would 
expect  since  assignment  was  random.  A  significant  correlation 
was  found  between  duty  position  and  P  Scores  (r=.38,  n=66,  pc.05) 
as  with  the  Multiple-Choice  Test  group;  however,  the  correlation 
between  duty  position  and  E  Scores  was  not  significant.  On  the 
other  hand,  reported  self  knowledge  was  significantly  correlated 
with  E  Scores  (r=.23,  n=74,  pc.05)  but  not  with  P  Scores.  Other 
correlations  were  only  marginally  greater  than  zero. 

IV.  Supervisor  Ratings  of  Performance 

For  the  purposes  of  the  present  research,  ratings  on  Scale  A 
(the  technical  skill  scale)  and  the  overall  soldier  effectiveness 
rating  are  most  relevant.  Additional  information  about 
supervisor  responses  to  the  other  scales  can  be  found  in  Appendix 
G » 
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A.  Descriptive  Statistics.  Supervisors  rated  the 
overwhelming  majority  (97%)  of  the  soldiers  technically  capable 
(Scale  A) .  Only  35%  of  76Cs  were  rated  competent  to  perform  all 
job  assignments  and  tasks  without  technical  assistance,  however. 
Scores  were  distributed  nearly  symmetrically  around  the  modal 
value  of  5  (moderate  skill) . 

In  terms  of  overall  effectiveness  (Scale  L) ,  supervisors 
rated  7%  of  the  soldiers  as  ineffective  and  unable  to  meet 
expectations  or  standards  of  performance  while  the  majority 
(64%)  were  rated  adequate  in  this  area.  The  remaining  29%  were 
rated  to  perform  excellently  and  exceed  standards  of  performance. 

In  an  effort  to  further  examine  the  supervisory  rating 
information,  the  data  were  broken  down  by  the  specific  posts  that 
participated.  Although  a  few  differences  were  found  between 
ratings  of  subordinates  at  different  sites,  these  differences 
were  neither  extreme  nor  fell  into  a  systematic  pattern. 

The  reliability  of  the  scale  items  was  assessed  through  the 
use  ofmthe  Cronbach  alpha  test.  The  overall  alpha  score  was 
.8950  fn=116)  and  all  the  individual  items  displayed  a  high 
degree  of  correspondence  with  the  overall  scale.  This  indicates 
either  a  lack  of  discrimination  by  the  rating  supervisors  in 
distinguishing  between  the  12  rating  categories  or  the  presence 
of  a  group  of  soldiers  each  of  whom  has  a  uniform  achievement 
across  a  very  broad  array  of  categories  each  of  which  is 
important  for  effective  soldiering. 


Table  3 

Correlation  of  Test  Scores  with  Supervisor  Rating  of  Technical 
Skill 


Test  Component 

Correlation 

Multiple-Choice 

0.12 

In-Basket  Forms 

O 

• 

o 

(Errors) 

Prioritizing 

0.08 

B.  Test  Scores 

and  Supervisory  Ratinas 

1.  Overview.  The  76C  MOS  has  an  unusual  structure  of 
first-line  supervision  as  was  previously  described  above.  Since 
neither  the  nominal  76Y  supervisor  or  the  motor  pool  sergeant 
( 63B)  has  technical  training  in  the  76C  subject  matter,  the 
accuracy  of  supervisory  performance  ratings  is  of  particular 
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interest.  Of  course,  in  a  typical  test  development  situation, 
supervisor  ratings  are  often  used  as  criteria  to  validate  the 
tests.  Under  the  present  circumstances,  it  was  unclear  what 
effect  would  be  produced  by  the  lack  of  any  systematic  technical 
training  given  to  the  76Y  or  63B  supervisor  on  76C  tasks.  Thus, 
the  present  effort  was  forced  to  put  particular  reliance  on  the 
content  validity  of  the  test  materials  pending  some  assessment  of 
the  utility  of  supervisory  ratings  in  this  situation. 

As  was  noted,  supervisor  ratings  of  76C  subordinates  were 
collected  on  a  number  of  performance  dimensions.  Then, 
correlations  between  these  ratings  and  the  76C  test  scores  were 
measured.  It  should  be  noted  that  not  all  of  the  soldiers  who 
took  the  Multiple-Choice  test  had  supervisors  participating  in 
this  research.  Generally,  supervisor  ratings  were  obtained  for 
only  about  50%  of  the  soldiers. 

2.  Multiple-Choice  Test  by  Supervisor  Ratings.  For  the 
group  of  soldiers  taking  the  Multiple-Choice  Test,  a  non¬ 
significant  correlation  (r=  0.12,  n=38)  was  obtained  between 
Testscores  and  supervisory  ratings  of  technical  skill  (Scale  A 
only) .  Interestingly,  the  correlation  between  Testscores  and  the 
performance  ratings  summed  across  all  12  rating  scales  was  -0.11. 
In  fact,  the  correlation  between  Testscores  and  supervisor 
ratings  had  its  maximum  when  only  Scale  A  ratings  were  used  and 
progressively  decreased  as  additional  scales  (dealing  with 
factors  other  than  technical  skill)  were  added  (see  Table  3) . 

A  similar  pattern  of  results  was  obtained  between 
Autoscores  and  the  supervisor  ratings.  A  breakdown  by  site 
also  showed  no  significant  deviations  from  these  results  at 
individual  posts. 

3.  In-Basket  Test  Scores  and  Supervisor  Ratings.  As  with 
the  Multiple-Choice  Test,  it  was  important  to  determine  the 
degree  of  correlation  of  supervisory  ratings  with  the 
corresponding  In-Basket  Test  scores.  For  the  overall  group  of 
soldiers  taking  the  In-Basket  Test,  a  correlation  of  -0.04  (n=41) 
was  obtained  between  E  Score  totals  and  supervisory  ratings  on 
Scale  A  (Technical  Knowledge/Skill)  and  a  correlation  of  0.08 
(n=36)  was  found  between  P  Score  Totals  and  Scale  A  ratings.  As 
with  the  Multiple-Choice  Test,  correlations  of  P  Scores  with  the 
sum  of  all  12  rating  scales  were  lower  (-0.05)  than  with  Rating 
Scale  A  only.  The  E  Score  correlation  with  the  sum  of  all  rating 
scales  was  -0.01  was  virtually  the  same  however  as  the 
correlation  with  Rating  Scale  A.  Site  differences  could  not  be 
assessed  reliably  due  to  the  small  Ns  at  Forts  Bragg  and  Riley 
(see  Table  3) . 
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DISCUSSION 


I .  Test  Battery  Methodology  and  Quality 

The  present  study  has  sought  to  lay  out  a  systematic  plan 
for  performance  evaluation  of  76C  AIT  and  field  performance, 
document  the  development  of  a  test  battery  created  according  to 
its  precepts  and  provide  an  initial  empirical  demonstration  of 
its  utility.  The  76C  MOS,  because  of  the  task  structure  and  the 
typical  field  conditions  for  performance,  presents  certain 
difficulties  in  assessing  performance  in  training  or  on  the  job. 
Nevertheless,  by  following  procedures  like  those  outlined  in  the 
Methodology  for  76C  Test  Construction,  assessment  accuracy  can  be 
increased. 

The  key  points  of  the  Methodology  are:  1)  the  development  of 
a  test  plan  in  which  a  matrix  maps  the  critical  functions  and 
task  operations  of  the  job,  2)  the  creation  of  items  which 
systematically  represent  each  of  the  test  plan  categories  (matrix 
cells) ,  3)  the  pilot  testing  of  items  to  ensure  their 
discriminative  capacity,  and  4)  the  use  of  consistent  scoring 
parameters . 

In  keeping  with  the  TTTA  concepts,  ARI  created  a 
demonstration  Test  Battery  for  the  PLL  Annex  of  the  76C  AIT 
course  at  Fort  Lee  following  the  Methodology  described  in  this 
report.  The  two  main  elements  of  the  Test  Battery  are  the 
Multiple-Choice  Test  and  the  In-Basket  Test.  The  pilot  study 
which  was  conducted  provided  a  field  test  of  the  acceptability 
of  the  tests  to  the  76C  journeymen  population  (face  validity)  as 
well  as  data  on  the  psychometric  properties  of  the  test  items  and 
item  types  used  in  the  Battery. 

The  statistical  analysis  of  the  Multiple-Choice  Test  showed 
that  the  test  items  displayed  appropriate  psychometric  properties 
in  pass-fail  rates,  range  of  difficulty,  and  item-test  score 
correlations  given  the  constraints  of  the  certification  process 
mandated  by  TRADOC  and  QMS.  As  noted,  these  constraints  mainly 
involve  the  required  coverage  of  certain  topics  irrespective  of 
their  difficulty.  Thus,  a  higher  proportion  of  items  than 
ordinary  have  pass  rates  in  excess  of  90%.  This  circumstance  may 
necessitate  a  longer  multiple-choice  test  than  that  used  in  this 
study,  preferentially  adding  optional  items  of  moderate  and  high 
difficulty. 

Internal  consistency  (reliability)  estimates  for  the 
Multiple-Choice  Test  were  in  the  moderate  range  (.48  adjusted). 
There  are  several  possible  reasons  for  this  finding.  First  as 
noted,  the  current  version  of  the  test  is  probably  too  short  at 
35  items  given  the  topic  requirements  of  TRADOC  and  QMS.  Longer 
tests  generally  have  higher  reliability  estimates  than  shorter 
tests.  If  the  number  of  test  items  was  doubled,  then,  according 
to  the  Spearman-Brown  formula,  the  internal  consistency  would  be 
increased  to  .65. 
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Second,  since  the  item  statistics  themselves  were  adequate 
with  few  exceptions,  this  low  measure  of  internal  consistency 
suggests  that  the  test  is  measuring  several  skills  or  knowledges 
which  are  not  highly  interrelated  at  least  as  far  as  the  subject 
population  is  concerned.  Domain  heterogeneity  has  a  direct 
effect  on  the  internal  consistency  measure  since  it  reduces  the 
likelihood  that  a  person  who  knows  some  given  topic  covered  by 
the  test  will  also  know  other  test  topics.  An  extreme  example  of 
domain  heterogeneity  might  be  a  test  which  contained  questions  on 
food  preparation  and  electronics.  By  using  a  matrix  taxomony  to 
construct  test  items,  knowledge  of  seven  tasks  is  sampled.  It  is 
very  plausible  that  soldiers  were  not  equally  proficient  in  each 
of  these  tasks. 

In  the  present  case,  some  level  of  soldier  heterogeneity  is 
likely  to  be  present  since  divergences  in  76Cs  training  and 
experience  occurs  after  AIT,  during  rotational  assignments.  For 
example,  assignment  to  a  combined  PLL  motor  pool  unit  leads  to  a 
daily  work  routine  quite  different  from  a  PLL  clerk  in  a  medical 
unit.  Certain  activities  are  equally  probable  in  both 
situations  but  other  tasks  will  be  much  more  likely  in  one  or 
the  other  position.  Two  other  factors  create  additional 
complications:  frequent  rotational  assignments  of  76Cs  which  may 
or  may  not  involve  positions  with  similar  daily  routines  and 
post-specific  training  experiences  and  supply  procedures  which 
diverge  in  some  respects  from  the  materials  presented  in  AIT. 

An  analysis  conducted  on  the  Multiple-Choice  Test  results 
to  determine  whether  the  inclusion  of  items  concerning  DA  Form 
3318  actions  differentially  affected  PLL  clerks  who  were 
currently  working  in  automated  PLL  units  showed  no  significant 
differences  on  overall  test  scores  as  a  function  of  their 
inclusion  or  omission  from  the  test.  Indeed,  item  analyses 
showed  that  performance  on  the  DA  Form  3318  items  was  comparable 
to  the  pattern  seen  on  the  other  items  for  the  group  as  a  whole. 
However,  the  post  AIT  training  and  job  experiences  of  this  group 
are  quite  varied  so  the  heterogeneity  may  reside  at  the  level  of 
the  individual  76C  rather  than  at  a  group  level. 

If  these  differences  in  post  AIT  experience  are  affecting 
the  levels  of  internal  consistency,  lengthening  the  test  may  not 
boost  the  reliability  coefficient  as  much  as  would  normally  be 
expected.  This  is  because  additional  items  would  probably 
maintain  or  increase  the  existing  level  of  perceived  or  actual 
domain  heterogeneity. 

As  a  final  note  on  the  matter  of  reliability,  what  has  been 
presented  thus  far  concerns  reliability  in  a  norm- referenced 
framework.  A  more  meaningful  measure  of  test  reliability  for  the 
military,  which  uses  criterion-referenced  tests,  is  a  measure  of 
how  consistently  the  test  can  measure  a  soldier  meeting  a  preset 
criterion  rather  a  specific  score.  A  measure  of  this  sort  of 
reliability  has  been  suggested  by  Linderman  and  Merenda  (1979)  in 
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which  computations  are  made  upon  the  number  of  students  reaching 
the  criterion  on  either  the  first,  second,  both  or  neither 
administrations  of  the  test. 

Finally,  it  should  be  noted  that  multiple-choice  tests  in 
general  for  this  MOS  will  place  a  greater  emphasis  on  reading 
comprehension  skills.  A  significant  portion  of  the  knowledge 
base  is  present  in  Army  supply  documents  and  technical  manuals 
which  require  a  high  school  reading  level  on  the  part  of 
personnel.  Although  it  would  have  been  useful  to  examine  test 
score-ability  correlations,  privacy  considerations  make  it 
difficult  to  conduct  analyses  of  76C  journeyman  test  performance 
as  a  function  of  personal  information  such  as  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB)  scores  etc. 

The  statistical  analysis  of  the  In-Basket  Test  results 
showed  that  all  but  one  of  the  10  f ill-in-the-form  (E  Score) 
items  displayed  acceptable  psychometric  properties  in  pass-fail 
rates,  range  of  difficulty,  and  item-test  score  correlations. 
Because  of  the  extended  number  of  responses  required  in  this  item 
type,  there  was  much  less  problem  with  ceiling  effects  for  item 
performance. 

The  internal  consistency  (reliability)  estimates  for  the  In- 
Basket  Test  E  Score  items  were  substantially  greater  than  for  the 
Multiple-Choice  Test  (.67  adjusted),  although  both  cover 
essentially  the  same  task  functions.  The  E  items  score  provide 
a  greater  degree  of  correspondence  with  the  corresponding  on-the- 
job  task  actions  than  the  multiple-choice  item  type,  which  may 
account  for  in  part  for  the  higher  internal  consistency  estimate. 

This  correspondence  has  its  good  and  bad  aspects  with 
respect  to  assessing  76C  job  knowledges  and  skills.  On  the 
positive  side,  soldiers  are  more  comfortable  with  this  item  type, 
whereas  on  the  negative  side  it  is  harder  to  score  reliably  and 
permits  more  rote  responding.  Perhaps  more  importantly,  student 
errors  on  these  items  do  not  permit  the  identification  of  the 
precise  areas  of  student  weakness.  Because  of  the  extended 
nature  of  the  items,  most  errors  have  multiple  potential  causes. 
Therefore,  the  overall  test  score  is  probably  far  more  meaningful 
than  the  interpretation  of  errors  for  any  given  item. 

In  addition  to  the  Fill-in  item  type,  the  In-Basket  Test  had 
four  rank  order  (P  Score)  items  which  required  soldiers  to  assign 
a  numerical  priority  to  each  of  a  series  of  job  actions  that  were 
to  be  performed  in  a  given  situation.  This  item  type  was  used 
in  order  to  measure  the  task  organizing  skills  of  76Cs,  which  as 
noted  in  the  Introduction,  has  been  one  area  of  criticism  of  76C 
field  performance.  One  potential  drawback  in  using  this  item 
type  is  that  such  skills  have  not  been  a  large  part  of  the  AIT 
curriculum,  although  presumably  76C  journeymen  should  have 
acquired  such  skills  during  their  time  on-the-job. 


26 


Soldiers  performance  on  this  item  type  was  low  in  terms  of 
pass-fail  rates  on  the  individual  items  (38%)  although,  as  noted, 
the  absolute  scores  for  this  item  type  will  tend  to  overestimate 
the  actual  level  of  errors.  Consequently,  the  criterion  score 
for  passing  a  test  should  be  lower  for  this  item  type.  The 
pattern  of  errors  on  these  items  indicated  that  soldiers  in  many 
cases  were  fundamentally  unsure  of  the  relative  priority  of 
certain  job  actions  rather  than  simply  transposing  an  action  up 
or  down  one  step  from  the  appropriate  rank  order.  Nevertheless, 
the  estimates  of  internal  consistency  (.72)  were  higher  for  these 
items  than  for  the  other  two  item  types  employed  in  the  Test 
Battery  . 

Given  the  lack  of  specific  training  in  AIT  on  the  content 
for  this  item  type,  the  initial  item  statistics  collected  here 
provide  encouragement  that  this  item  type  may  provide  useful 
information  about  job  organizing  skills.  Training  materials 
developed  as  part  of  other  Fort  Lee  TTTA  efforts  offer  explicit 
instruction  and  testing  on  these  kind  of  skills  for  the  first 
time.  It  is  essential  that  76Cs  performance  on  these  item  types 
be  measured  again  when  such  training  becomes  part  of  their  AIT 
experience. 

II .  Job  History  Survey 

In  addition  to  the  two  tests,  the  76C  soldiers  completed  a 
short  Job  History  survey  form  which  provides  a  rough  picture  of 
the  kind  of  work  and  training  experiences  that  journeymen  PLL 
clerks  are  likely  to  have  received  during  the  period  1986-1989. 

In  contrast  to  previous  surveys  (e.g.,  Hughes  Aircraft  1982),  it 
now  seems  that  a  high  percentage  of  personnel  in  PLL  duty 
positions  have  gone  through  AIT  rather  than  being  reassigned  from 
other  MOSs.  Additional  training  or  retraining  is  still  common  in 
post  schools,  although  one  site  (Fort  Bragg)  has  discontinued 
their  formal  on-site  PLL  course.  Whether  the  increase  in  AIT 
experience  among  current  PLL  clerks  has  or  will  reduce  the  need 
for  such  courses  remains  an  unanswered  question  at  this  time. 

The  Job  History  survey  presented  an  opportunity  to  see  if 
particular  job  or  training  experiences  might  have  some 
correlation  with  test  performance.  This  analysis  of  the  survey 
responses  with  test  scores  in  fact  revealed  little  apparent 
connection  between  any  of  the  experience  and  training  variables 
and  scores  on  either  the  Multiple-Choice  or  In-Basket  Tests.  The 
one  substantial  positive  correlation  was  between  self-reports  of 
degree  of  job  knowledge  and  test  scores.  In  other  words, 
soldiers  had  a  fair  idea  of  how  much  they  knew  and  didn't  know 
about  PLL  procedures. 

The  absence  of  correlation  obtained  between  these  measures 
is  not  unique  to  this  study;  other  investigations  have  revealed  a 
similar  non-association  between  job  performance  measures  and  job 
experiences  (Schurman  et  al.,  1980).  One  possible  explanation  is 
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that  the  recent  task  experience  is  the  most  important  factor  in 
performance.  All  of  the  soldiers  had  been  or  currently  were  in 
the  PLL  duty  position  within  the  two  preceding  years.  At  least 
as  far  as  the  present  sample  is  concerned,  the  vast  majority  of 
active  PLL  clerks  have  been  through  AIT  at  QMS  which  provides 
some  level  of  uniformity  to  the  training  experiences  of  these 
personnel . 

Ill .  Supervisory  Ratings  and  Test  Scores 

Supervisors  of  the  soldiers  were  also  asked  to  provide 
performance  ratings  of  their  own  subordinates.  Only  one  of  the 
12  scales  dealt  directly  with  supervisor  perceptions  of  their 
subordinate's  technical  skill,  although  the  other  scales  did 
cover  other  aspects  of  soldier  effectiveness. 

Supervisor  ratings  of  their  subordinate's  technical  skill 
showed  very  low  correlations  with  the  test  scores  of  these 
personnel.  However,  it  should  be  noted  that  the  combined  ratings 
from  all  12  scales  had  a  significantly  lower  (and  negative) 
correlation  with  test  scores  than  did  Scale  A  alone.  In  fact, 
the  correlation  steadily  decreased  as  additional  scales  were 
added  to  Scale  A  ratings.  This  pattern  of  results  suggests  that 
supervisors  were  distinguishing  the  different  scales  in  terms  of 
their  ratings  and  that  they  were  probably  considering  more  skill 
related  behaviors  while  answering  Scale  A  than  for  the  other 
scales.  However,  it  also  indicates  that  these  supervisors  do  not 
seem  to  have  a  clear  awareness  of  the  technical  skills  of  their 
subordinates. 

It  might  be  argued  that  supervisors  do  have  an  accurate  idea 
of  thexr  subordinates'  technical  skills  and  the  Test  Battery  is 
not  providing  an  accurate  index  of  their  skills.  Several  points 
argue  against  this  possibility.  First,  the  materials  and  topics 
being  tested  in  both  the  Multiple-Choice  Test  and  the  In-Basket 
Test  come  straight  from  the  AIT  course  materials  as  well  as  the 
major  Army  supply  documents  and  regulations.  Second,  the  In- 
Basket  Test  provides  a  close  approximation  of  the  actual  tasks 
performed  by  the  PLL  clerks.  Third,  soldiers  raised  no 
complaints  about  the  items  being  hard  to  understand  or  outside 
their  experience  and  training.  The  one  small  exception  to  this 
were  the  multiple-choice  items  which  dealt  with  DA  Form  3318 
since  this  is  used  only  where  automated  PLL  procedures  are 
absent.  Nevertheless,  even  in  this  cas^,  all  soldiers  agreed 
that  they  had  received  training  in  the  manual  PLL  procedures  and 
that  they  had  to  know  it  for  their  SQT. 

It  should  be  noted  that  a  confounding  does  exist  since 
supervisors  rated  only  their  own  subordinates.  Thus  it  is  not 
possible  to  control  for  response  differences  between  raters. 
Nevertheless,  an  examination  of  the  ratings  did  not  indicate 
high  levels  of  response  bias. 
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One  unusual  systemic  feature  of  the  76C  MOS  is  the 
relationship  of  supervisor  to  subordinate.  Since  the  76C  series 
ends  at  the  rank  of  Sergeant,  promotions  past  this  point  are  to 
the  76Y  (Unit  Supply  Specialist)  MOS.  Thus,  in  most  cases,  the 
nominal  supervisor  of  a  76C  is  a  76Y  who  is  not  co-located  with 
his  subordinate  7 60s.  More  than  two-thirds  of  the  76Y  NCOs  have 
always  been  assigned  to  that  MOS  and  have  no  specific  experience 
as  a  PLL  clerk  or  even  a  76C.  Because  of  this  anomaly,  in  many 
cases  the  actual  rating  supervisor  is  a  63B  Motor  Pool  Sergeant 
who  has  received  no  76  series  training  either  a  76Y  or  76C.  On 
the  other  hand,  the  63B  is  at  least  co-located  with  the  76C  PLL 
clerk  and  interacts  with  him/her  on  a  daily  basis.  One 
additional  complication  occurs  in  combined  PLL  motor  pools  which 
have  a  common  PLL  section  for  a  number  of  different  companies. 

In  these  situations,  since  there  is  a  group  of  PLL  clerks  working 
together,  a  division  of  tasks  may  exist,  some  clerks  may  "cover" 
for  others,  etc.  All  these  factors  make  the  supervisor's  task  of 
assessing  individual  technical  skill  more  difficult.  Thus, 
unlike  most  work  settings,  the  76C  supervisor,  whether  76Y  or 
63B,  may  not  be  the  best  judge  of  their  subordinates'  technical 
skill. 

In  summary,  a  systematic  methodology  for  assessment  of  76C 
AIT  and  journeyman  performance  has  been  presented  along  with  a 
first  ;tage  implementation  of  the  plan  in  the  form  of  an 
integrated  Test  Battery.  Both  the  Multiple-Choice  Test  and  the 
In-Basket  Test  displayed  acceptable  item  statistics  although  the 
Multiple-Choice  Test  should  probably  be  expanded  to  50-60  items 
provide  a  consistent  method  for  measuring  76C  AIT  performance 
changes  over  time  as  a  function  of  new  training  procedures 
instituted  in  the  course.  It  also  provides  a  yardstick  for 
school  based  efforts  tc  improve  the  accuracy  of  evaluations  of 
student  performance  and  knowledge  acquisition. 

CONCLUSIONS 


A  systematic  methodology  for  assessing  76C  AIT  and 
journeyman  performance  is  now  availaole  along  with  a  prototype 
test  battery. 

A  general  model  for  developing  content-valid,  i.e.  job 
relevant  tests  is  also  available  and  is  applicable  to  any  MOS. 

The  components  of  the  prototype  (multiple-choice  and  in- 
basket  tests)  are  reliable  but  the  multiple-choice  test  can  be 
improved  by  expanding  it  from  35  to  50  items.  A  criterion- 
referenced  measure  of  test  reliability  is  recommended 

A  revised  test  battery  will  provide  a  consistent  method  for 
measuring  76  AIT  performance  changes  over  time  as  a  function  of 
new  training  procedures  instituted  in  the  course. 
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The  manual  PLL  performance  of  76Cs  in  units  is  adequate 
given  the  increasing  use  of  computer  aiding.  However,  the  higher 
order  skill  of  prioritizing  tasks  is  not  currently  mastered  by 
these  personnel. 
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Appendix  A 
76C  Critical  Tasks 
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APPENDIX  A 


BSEP  Critical  Tasks  for  76C  MOS 


1. 

76C 

101-539-1101 

2. 

76C 

101-539-1102 

3. 

76C 

101-539-1103 

4. 

76C 

101-539-1104 

5. 

76C 

101-539-1105 

6. 

76C 

101-539-1106 

7. 

76C 

101-539-1107 

8  . 

76C 

101-539-1108 

9. 

76C 

101-539-1109 

10. 

76C 

101-539-1110 

Maintain  a  Prescribed  Load  List  (Manual) 

Prepare  and  Maintain  Non-stock  List 
Records 

Process  a  Request  for  a  Prescribed  Load 
List  Repair  Part  (Manual) 

Process  a  Request  for  a  Prescribed  Load 
List  Repair  Part  (Automated) 

Process  a  Request  for  Non-stockage  List 
Repair  Part  (Manual) 

Process  a  Request  for  Non-stockage  List 
Repair  Part  (Automated) 

Process  a  Request  for  Non-national  Stock 
Number  Repair  Part  (Manual) 

Process  a  Request  for  Non-national  Stock 
Number  Repair  Part  (Automated) 

Process  a  Request  for  a  Repair  Part 
Designated  as  a  Direct  Exchange  (Manual) 

Process  a  Request  for  a  Repair  Part 
Designated  as  a  Quick  Supply  Store 
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11. 

12. 

13. 


14. 


15. 


16. 


17. 


16. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


76C  101-539-1111 
76C-101-539-1112 
76C-101-539-1113 
76C-101-539-1114 
76C-10 1-539-1 115 

76C-101-539-1116 

76C-101-539-1118 
760101-539-1119 
76C-1 01-539-1 120 
760101-535-1121 
760101-539-1301 

7 6C-1 01-539- 1302 

760101-535-1303 

7 6C-1 01-539-1304 

760101-539-1305 

760101-539-1306* 

760101-539-1307 


Receive  Repair  Farts  (Manual) 

Receive  Repair  Parts  (Automated) 

Turn-in  Repair  Parts  (Manual) 

Turn-Repair  Parts  (Automated) 

Conduct  Review  and  Inventory  of  Prescribed  Load 
List  (PLL)  Records  (Manual) 

Process  Prescribed  Load  List  (PLL)  Change  Listings 
Au  toma ted ) 

Process  Supply  and  shipment  Status  List  of  Cards 

Initiate  Follow-up  or  Document  Modification  Action 

Initiate  Cancellation  Action 

Perform  Reconciliation  of  Due-in 

Prepare  and  Maintain  an  Equipment  Log  MOS:  17K 

54E 

Prepare  and  Maintain  a  Preventive  Maintenance 
Schedule  and  Record 

Prepare  and  Maintain  an  Equipment  Uncorrected  Fault 
Record 

Request  Repair  or  Modification  of  Equipment 
MOS :  31M 

Prepare  and  Maintain  an  Equipment  Component  Register 
Prepare  an  Equipment  Control  Record  MOS:  54E 
Dispatch  and  Record  Return  of  Equipment  MOS:  76C 
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28.  76C— 101— 539— 1 308  Prepare  and  Maintain  The  Material  Condition  Status 

Report 

29.  76C-101-539-1309  Prepare  and  Maintain  the  Oil  Analysis  Oil 

30.  76C-101-539-1401  Process  Complete  or  Rejected  Request  for  Maintenance 

31.  76C-101-539-1402  Process  Maintenance  Work  Request  Envelope  and  Update 

Maintenance  Workload  Status 

32.  76C-101-539-1403  Process  Complete  or  Rejected  Request  for 

Main  tenance 

33.  76C-101-539-1404  Process  an  Intra-Shop  Work  Request 

34.  76C-101-539-1405  Process  a  work  Request  in  Shop  Supply 

35.  76C-101-539-1406  Maintain  a  Shop  Stock  List 

36.  76C-101-539-1407  Maintain  a  Bench  Stock 

37  76C-101-539-1408  Prepare  and  Maintain  a  Parts  Request/ Status 

Register  for  a  Request  (Manual) 

36.  76C-101-539-1409  Prepare  and  Maintain  a  Parts  Request/Status 

Register  for  Status  (Automated) 

39.  76C-101-539-1410  Conduct  Review  of  Shop  Stock  Records  (Manual) 
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Appendix  B 

Component  Elements  of  76C  Critical  Tasks 


(Available  upon  Request  from  the  Author) 


Appendix  C 

Prerequisite  Competencies  for  76C  Critical  Tasks 
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TABLE  1:  Prerequisite  Baseline  Skills 

1  NUMBERING  AND  COUNTING 

1A  MATCH  NUMERALS  WITH  WORD  NAMES  AND  MODELS 

IB  WRITE  NUMERALS  ONE  THROUGH  N  IN  SEQUENTIAL  ORDER  FROM  ANY 
STARTING  POINT 

1C  STATE  WHAT  NUMERAL  COMES  AFTER,  BEFORE,  OR  BETWEEN  ANY  TWO 
GIVEN  NUMBERS 

ID  SELECT  THE  NUMERAL  WHICH  IS  GREATER/ LESSER  FROM  A  SET  OF 
NUMERALS 

IE  IDENTIFY  AN  OBJECT  WITH  A  SPECIFIED  ORDINAL  POSITION 

IF  WRITE  OR  STATE  THE  PLACE  VALUE  OF  A  PARTICULAR  DIGIT, 

WHOLE  OR  DECIMAL  NO. 

1G  ROUND  OFF  A  NUMBER  TO  A  SPECIFIED  PLACE,  WHOLE  OR  DECIMAL 

1H  COUNT  BY  ONES,  TWOS,  FIVES,  TENS,  ETC.,  BACKWARD  OR 
FORWARD  (SKIP  COUNTING) 

II  MATCH  NUMBERS  WITH  POINTS  OR  INTERVALS  ON  A  NUMBER  LINE 
(POS  -OR  NEG  VALUES) 

2  LINEAR,  WEIGHT,  AND  VOLUME  MEASURES 

2A  NAME  THE  MARKINGS  ON  A  LINEAR  SCALE 

2B  DIFFERENTIATE  UNITS  OF  MEASURE  &  EQUIVALENTS  IN  THE 
ENGLISH  &  METRIC  SYSTEM 

2C  USE  A  RULER,  YARDSTICK,  METER  STICK  OR  SCALE  TO  MEASURE 
LENGTH  OR  DISTANCE 

2D  IDENTIFY  MEASURE  OF  OUNCE,  POUND,  OR  GRAM 

2E  IDENTIFY  MEASURES  OF  PINTS,  QUARTS,  GALLONS,  LITERS 

2F  USE  A  SCALE  WHICH  IS  NOT  NUMERICALLY  CALIBRATED 

2G  ESTIMATE  MEASURES  OF  VARYING  LENGTHS,  DIMENSIONS  OR 
WEIGHTS 

3  DEGREE  MEASURES 
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3A  IDENTIFY  DEGREE  OR  MIL  AS  A  UNIT  IN  DETERMINING  DIRECTION, 
DISTANCE  OR  TEMP 

3B  ESTIMATE  THE  MEASURE  OF  A  GIVEN  ANGLE  NOT  GREATER  THAN  180 
DEGREES 

3C  INTERPRET  BEAR INGS , AZ IMUTH  ETC.,  IN  WHICH  ANGLES  RANGE  0-360 
DEG/0-6400  MILS 

4  TIME-TELLING  MEASURES 

4A  USE  A  24-HOUR  OR  DIGITAL  CLOCK  TO  TELL  TIME 

4B  NAME  INTERVALS  AND  TELL  TIME  IN  HOURS,  MINUTES,  AND  SECONDS 

4C  ESTIMATE  TIME  IN  SECONDS,  MINUTES,  AND  PARTS  OF  AN  HOUR 

4D  IDENTIFY  CALENDAR  UNITS  AND  ARRANGE  THEM  IN  JULIAN  STYLE 

4E  CONVERT  TIME  INTO  HOURS  AND  TENTHS  OF  HOURS 

4F  COMPUTE  TIME  -  USE  GREENWICH  MEAN  TIME  (GMT)  TO  ESTABLISH 
ZONES  &  DISTANCES 

5  GAUGE  MEASURES 

5A  IDENTIFY  THE  UNIT  OF  MEASUREMENT  FOUND  ON  AN  INSTRUMENT 

5B  INTERPRET  THE  NUMBER,  WORD,  SYMBOL  FROM  A  DISPLAY  READ-OUT 

5C  RECOGNIZE  A  "READING"  FROM  A  GAUGE  WITH  COLOR  DIVISIONS 

5D  RECOGNIZE  POSITIVE  <+)  AND  NEGATIVE  (-)  DEMARCATION  ON  A 
SCALE 

5E  SELECT  BAND(S)  FROM  A  MULTI-SCALE  GAUGE 

5F  MATCH  A  GAUGE  READING  TO  A  SPECIFICATION  USING  NUMBERED, 
LABELED  INTERVALS 

5G  INTERPRET  GAU&E  READINGS  FROM  AN  UNNUMBERED/UNMARKED 
INTERVAL 

5H  INTERPRET  A  GAUGE  READING  WHICH  IS  FLUCTUATING  OR 
MOMENTARILYY  SUSTAINED 

51  MATCH  SPECS  OF  REQUIRED  MEASURES  BY  MANIPULATION,  ALIGNMENT 
OR  MAINTENANCE 
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6  SPATIAL 


6A  IDENTIFY  DIRECTIONS  THAT  TOOLS ,  HARDWARE,  OR  COMPONENTS  MAY 
BE  MOVED 

6B  MANIPULATE  OBJECTS  TO  ALIGN,  MATCH,  MATE,  BE  PARALLEL, 
PERPENDICULAR,  ANGLED 

6C  INTERPRET  SPATIAL  RELATIONSHIPS  OF  FIGURES  AND  OBJECTS  FROM 
2-D  REP 

6D  RELATE  GEOMETRIC  SYMBOLS  AND  GRAPHIC  REPRESENTATIONS  TO 
ACTUAL  SYSTEMS,  ETC 

7  LINES 

7 A  IDENTIFY  AND  NAME  POINTS,  LINES,  RAYS,  AND  SEGMENTS 

7B  IDENTIFY  INTERSECTING  LINES,  PARALLEL  LINES,  AND  LINE 
SEGMENTS 

7C  DEFINE  AND  IDENTIFY  PERPENDICULAR  LINES 

7D  IDENTIFY  CONGRUENT  SEGMENTS 

8  PLANES 

8A  IDENTIFY  AND  NAME  PLANE  GEOMETRIC  FIGURES 

8B  LIST  THE  CHARACTERISTICS  OF  GEOMETRIC  FIGURES 

8C  CLASSIFY  FIGURES  ACCORDING  TO  THE  NUMBER  OR  MEASURE  OF  THEIR 
THEIR  SIDES  OR  ANGLES 

80  IDENTIFY  FIGURES  WHICH  POSSESS  SIMILARITIES 

8E  IDENTIFY  FIGURES  WHICH  MAY  BE  PARALLEL,  PERPENDICULAR  OR 
CONGRUENT 

9  ANGLES  AND  TRIANGLES 

9A  IDENTIFY,  NAME  AND  SYMBOLIZE  THE  DIFFERENT  KINDS  OF  ANGLES  A 
AND  TRIANGLES 

9B  IDENTIFY  VERTICAL,  ADJACENT,  COMPLEMENTARY  OR  SUPPLEMENTARY 
ANGLES 


C-4 


9C  CLASSIFY  TRIANGLES  ACCORDING  TO  THEIR  SIDES  OR  ANGLE-SIZE 

9D  IDENTIFY  ALTITUDES  AND  MEDIANS  OF  TRIANGLES  OR  THE  BISECTOR 
OF  AN  ANGLE 

9E  NAME  AN  ANGLE  BY  USING  LETTERS#  A  NUMBER#  OR  A  SINGLE 
LETTER 

10  SOLIDS 

1 OA  RECOGNIZE  AND  MATCH  THE  NAMES  OF  SOLIDS  WITH  THEIR  CORRESPON 
CORRESPONDING  FIGURES 

11  TERMINOLOGY 

1  1 A  IDENTIFY  TECHNICAL  WORDS  ASSOCIATED  WITH  GEOMETRIC  FIGURES 
1  1 B  INTERPRET  MEANING  OF  TERMS  DERIVED  FROM  SPATIAL  ORIENTATION 

12  ADDITION  AND  SUBTRACTION 

12A  ADD  OR  SUBTRACT  WHOLE  NUMBERS.  WITHOUT  CARRYING  OR  BORROWING 

12B  ADD  OR  SUBTRACT  WHOLE  NUMBERS.  CARRYING  AND  BORROWING 

12C  ADD  &  SUBTRACT,  BORROWING  &  CARRYING  WITH  MIXED  NUMBERS 
(WHOLE  AND  DECIMAL) 

1 2D  ADD  OR  SUBTRACT  POSITIVE  &  NEGATIVE  NUMBERS  USING  A  NUMBER 
LINE  FOR  SOLUTION 

12E  ADD  OR  SUBTRACT  TO  FIND  CORRECT  TIME  (2A  HR.  CLOCK)  USING 
HOURS  OR  MINUTES 

12F  ADD  OR  SUBTRACT  VARIOUS  INCREMENTS  ON  GAUGES#  DIALS,  OTHER 
MEAS.  INSTRUMENTS 

1 26  ADD  OR  SUBTRACT  TIME .LINEAR , DRY, LIQUID  OR  DEGREE  MEASURES 
REQ.  REGROUPING 

12H  ESTIMATE  A  SUM  OR  DIFFERENCE 

13  MULTIPLICATION  AND  DIVISION 

13A  MULTIPLY  AND  DIVIDE  WHOLE  NUMBERS 

13B  MULTIPLY  AND  DIVIDE  MIXED  NUMBERS  (WHOLE  AND  DECIMALS) 
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13C  DIVIDE  A  NUMBER  WITH  DECIMALS  IN  BOTH  DIVISOR  AND  DIVIDEND 

1 3D  MULT  &  DIV  INTEGERS,  BOTH  POSITIVE  &  NEGATIVE  &  ASSIGN 
PROPER  SIGN  TO  RESULT 

13E  ESTIMATE  A  PRODUCT  OR  QUOTIENT 

14  FRACTIONS/DECIMALS 

14A  SUBDIVIDE  WHOLE  OBJECTS  OR  A  SET  OF  OBJECTS  INTO  1/2’S, 

1/3’S,  1/4’S,  1/8’S 

14B  REDUCE  FRACTIONS  TO  LOWEST  TERMS 

14C  CONVERT  FRACT.  (PROPER  &  IMPROPER)  TO  DEC.  EQUIV  &  VICE 
VERSA  USING  TABLE 

14D  COMPUTE  EQUIVALENT  VALUE  OF  FRACT. ,DEC. , PERCENTS, &  MIXED 
NOS.  TO  LOWEST  TERMS 

14E  ADD  AND  SUBTRACT  FRACTIONS,  WITH  SAME  OR  DIFFERENT 
DENOMINATIONS 

14F  MULTIPLY  AND  DIVIDE  FRACTIONS  WITH  AND  WITHOUT  WHOLE  NUMBERS 

14G  ESTIMATE  A  FRACTIONAL  SUM,  PRODUCT,  OR  QUOTIENT 

15  GEOMETRY 

15A  DRAW  GEOMETRIC  FIGURES,  PLANE  AND  SOLID 

15B  MATCH  GEOMETRIC  FIGURES  WITH  WORD  NAMES,  EQUIVALENT  MEASURES 

15C  LABEL  GEOMETRIC  FIGURES  USING  MATHEMATICAL  AND  CHARACTERISTIC 
DESIGNATORS 

15D  USE  A  PROTRACTOR  TO  MEASURE  ANGLES,  MAKE  GEOMETRICAL 
CONSTRUCTIONS 

15E  CONSTRUCT  PERPENDICULAR  ON  A  LINE  SEGMENT,  BISECTOR  OF  AN 
ANGLE 

15F  COMPUTE  THE  PERIMETER  AND  AREA  OF  ANY  FIGURE 

15G  COMPUTE  THE  CIRCUMFERENCE  AND  AREA  OF  A  CIRCLE 

15H  COMPUTE  THE  AREA  AND  VOLUME  OF  ANY  SOLID  FIGURE 
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151  USE  FORMULAS  IN  SOLVING  PROBLEMS  INVOLVING  GEOMETRIC  FIGURES 


1 5 J  SOLVE  PROBLEMS  &  INTERPRET  SPATIAL  RELATIONS  OF  FIGURES ,  ETC 
FROM  2-D  REP 

16  COMBINATION  OF  PROCESSES 

1 6 A  IDENTIFY  MEDIAN  AND  MODE 
16B  COMPUTE  AVERAGES 

16C  SOLVE  PROBLEMS  COMBINING  ALL  PROCESSES  USING  WHOLE,  MIXED 
NOS.  AND  FRACTIONS 

16D  SOLVE  PROBLEMS,  COMBINING  ALL  PROCESSES,  INVOLVING  UNITS  OF 
MEASUREMENT 

16E  INTERPRET  INFO  FROM  CHARTS , NUMBER  LINES, SCALES  &  GRAPHS  TO 
SOLVE  ARITH  PROB 

16F  SOLVE  CONVERSION  PROBLEMS  OF  LINEAR,  LIQUID,  WEIGHT  AND 
TEMPERATURE  MEASURES 

16G  SOLVE  PROBLEMS  INVOLVING  RATIO  AND  PROPORTION 

16H  SOLVE  WORD  PROBLEMS  WHERE  ANY  MATHEMATICAL  PROCESS  MAY  OCCUR 

17  GRAPHING  IN  THE  COORDINATE  PLANE 

17A  IDENTIFY  COORDINATES  OF  A  POINT  IN  ANY  GRID  SYSTEM 
17B  IDENTIFY  POINTS  ON  A  LINE  GRAPH 
17C  MATCH  A  GRAPH  WITH  ITS  EQUATION 

18  ALGEBRA 

18A  SOLVE  SIMPLE  ALGEBRAIC  EQUATIONS  WITH  ONE  UNKNOWN 
18B  RECOGNIZE  ANQ  DERIVE  EQUIVALENT  ALGEBRAIC  EXPRESSIONS 
18C  EVALUATE  POWERS  AND  ESTIMATE  ROOTS 
1  9  TRIGONOMETRY 

19A  USE  TABLES  OF  TRIGONOMETRIC  FUNCTIONS 
19B  USE  TABLES  OF  LOGARITHMS  TO  SOLVE ' PROBLEMS 
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1 9C  SOLVE  GEOMETRIC  PROBLEMS  USING  TRIGONOMETRIC  FUNCTIONS 

19D  USE  TRIGONOMETRIC  RATIOS  TO  SOLVE  PROBLEMS 

25  PROCEDURAL  DIRECTIONS 

25A  IDENTIFY  FACTUAL  DETAILS  OR  SPECIFICATIONS  FOUND  WITHIN  A 
STATEMENT 

25B  SELECT  PARTS  OF  TEXT  AND  VISUAL  MATERIALS  TO  COMPLETE  A  TASK 
ACTIVITY 

25C  FOLLOW  HIGHLY-DETAILED  DIRECTIONS  TO  ACCOMPLISH  SEQUENCE  OF 
TASK  ACTIVITIES 

25D  DETERMINE  THE  ESSENTIAL  MESSAGE  OF  A  PARAGRAPH  OR  SECTION  OF 
WRITTEN  MAT '  L 

25E  INFER  FROM  A  WRITTEN  SOURCE,  WHERE  INFO  IS  NOT  EXPLICIT,  TO 
MAKE  A  DECISION 

25F  SYNTHESIZE  INFO  FROM  WRITTEN  SOURCES  CONTRIBUTING  TO 
COMPLETION  OF  TASK 

26  VOCABULARY 

26 A  RECOGNIZE  COMMON  WORDS  AND  THEIR  MEANINGS 

26 B  RECOGNIZE  TASK-RELATED  WORDS  WITH  TECHNICAL  MEANINGS 

26C  IDENTIFY  THE  CORRECT  MEANING  OF  A  WORD  FROM  THE  CONTEXT  OF 
A  SENTENCE 

26D  RFCOGNIZE  THE  MEANING  OF  COMMON  CONTRACTIONS,  ABBREVIATIONS 
AND  ACRONYMS 

26E  DETERMINE  MEANING  OF  FIG,  IDIOM,  TECH  TERMS  USING  CONTEXT 
CLUES,  REFERENCES 

27  REFERENCE  SKILLS' 

27A  LOCATE  TM,  FM,  OR  ANY  RELATED  SOURCE  DOCUMENT  BY  CODE  NUMBER 
AND  TITLE 

27B  ALPHABETIZE  WORDS  OR  TOPICS  TO  LOCATE  INFORMATION 

27C  USE  TABLE  OF  CONTENTS,  INDEX,  HEADINGS,  APPENDIX  &  GLOSSARY 
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TO  LOCATE  INFO 


270  LOCATE  PAGE,  TITLE,  PARA,  FIG,  OR  CHARTS  TO  ANSWER  QUESTION/ 
SOLVE  PROBLEM 

27E  DETERMINE,  AFTER  SCANNING  OR  SKIM-READING,  WHETHER  THE 
INFO  IS  RELEVENT 

27F  XREF  W/IN  &  ACROSS  SOURCE  DOCUMENTS,  SELECT  INFO  NEEDED  TO 
PERFORM  ROUTINE 

27G  ORGANIZE  INFO  FROM  MULTIPLE  SOURCES  INTO  A  SEQUENCED  SERIES 
OF  EVENTS 

28  TABLES/CHARTS 

28 A  OBTAIN  A  FACT  OR  SPECIFICATION  FROM  A  2-COLUMN  TABLE  OR 
CHART  TO  FIND  INFO 

28B  OBTAIN  A  FACT  OR  SPEC  FROM  AN  INTERSECTION  OF  A  ROW  BY 
COLUMN  TABLE  OR  CHART 

28C  USE  COMPLEX  TABLE/CHART  REQUIRING  XREF  W/IN  CHART  OR  IN 
COMBW/OUTSIDE  TEXT 

28D  APPLY  INFO  FROM  TABLES  &  CHARTS  TO  LOCATE  MALFUNCTIONS/ 
SELECT  ACTION  COURSE 

29  ILLUSTRATIONS 

29A  IDENTIFY  DETAILS,  LABELS,  NUMBERS,  AND  PARTS  FROM  AN 
ILLUSTRATION  OR  PICTURE 

29B  IDENTIFY  PARTS  OR  DETAILS  ACCORDING  TO  A  KEY  OR  LEGEND 

29C  INTERPRET  A  CROSS-SECTIONAL  DRAWING  OF  AN  OBJECT  FOR 
ASSEMBLY  OR  DISASSEMBLY 

29D  INTERPRET  3-D  PROJ  OR  EXPLODED  VIEW  FOR  ASSEMBLY  OR  POSITION 
IN  SYS/SUBSYS 

29E  FOLLOW  ILLUSTRATIONS  OR  PHOTOGRAPHS,  ARRANGED  IN  A 
SEQUENTIAL  ORDER, AS  GUIDE 

29F  INTEGRATE  INFORMATION  FROM  VARIOUS  SOURCES  TO  SELECT  A 
COURSE  OF  ACTION 

30  FLOW  CHARTS 
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30 A  USE  A  SIMPLE  LINEAR  PATH  OR  ORGNIZATIONAL  CHART  TO  LIST 
EVENTS  IN  SEQUENCE 

30B  USE  A  LINEAR  PATH  OF  A  FLOW  CHART  TO  PROVIDE  VISUAL  AND 
TEXTUAL  DIRECTIONS 

30C  TRANSLATE  THE  SIGNIFICANCE  OF  THE  SYMBOLS  INTO  PHYSICAL 
ACTIVITIES 

31  SCHEMATICS 

3 1 A  ISOLATE  EACH  MAJOR  SECTION  OR  ENTITY  PRESENTED  IN  A 
SCHEMATIC  DIAGRAM 

3 1 B  IDENTIFY  THE  COMPONENTS  WITHIN  EACH  ENTITY 

3 1 C  TRACE  CONNECTIONS  IN  AN  INTEGRATED  CIRCUIT  FROM  ORIGIN  TO 
ANOTHER  POINT 

3 1 D  ISOLATE  A  PROBLEM  COMPONENT  IN  A  SCHEMATIC  &  TRACE  IT  TO 
CAUSING  COMPONENTS 

3 1 E  INTERPRET  SYMBOLS  TO  INDICATE  DIRECTION  OF  FLOW,  TEST 
POINTS,  COMPONENTS, ETC 

32  FORMS 

32A  LOCATE  THE  BLOCK  ON  A  FORM  TO  ENTER  THE  APPROPRIATE 
INFORMATION 

32B  TRANSFER  A  NO.,  DATE,  ETC.  FROM  EQUIP.  OR  WRITTEN  SOURCE  TO 
SECTION  OF  FORM 


32C  WRITE  NAME  OF  ORG.»  PERSONNEL,  DISPOSITION,  NOMENCLATURE  ETC 
IN  FORM  SECTION 

32D  WRITE  A  DESCRIPTIVE  ACCOUNT  OF  AN  ACTIVITY  OR  TRANSACTION 
PERFORMED 

32E  USE  A  COMPLEJED  FORM  TO  LOCATE  OR  COMPARE  INFORMATION 
33  NOTE-TAKING 

33A  DISTINGUISH  BETWEEN  ESSENTIAL  AND  NON-ESSENTIAL  DETAILS 
WHILE  NOTE-TAKING 

33B  RECORD  DETAILS  W/O  MISINTERPRETING  INTENT  OF  WRITTEN 
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MATERIAL  OR  INTERVIEW 


33C  REWRITE  ALL  RECORDED  DETAILS  IN  SENTENCE  FORM 
33D  ORGANIZE  ALL  SENTENCES  INTO  PARAGRAPHS 

34  OUTLINING  (TOPIC  OR  SENTENCE) 

34A  DISTINGUISH  BETWEEN  MAJOR  AND  SUBORDINATE  TOPICS 

34B  GENERATE  TITLES  FOR  EACH  MAJOR  TOPIC  SELECTED 

34C  USE  PHRASES/SENTENCES  TO  PROVIDE  SUBORDINATE  DETAILS  FOR 
EACH  MAJOR  TOPIC 

34D  ALTERNATE,  INDENT  NUMBERS  AND  LETTERS  TO  ESTABLISH  A 
HIERARCHY 

35  REPORT  WRITING 

35 A  STATE  THE  INTENT  OR  OBJECTIVE(S)  OF  THE  REPORT 

35B  DESCRIBE  THE  PARAMETERS  OF  THE  EVENT  OR  SITUATION 

35C  DISTINGUISH  BETWEEN  RELEVANT  AND  IRRELEVANT  DETAILS 

35D  SEQUENCE  EVENTS  IN  THE  ORDER  THEY  HAVE  OCCURRED 

35E  STATE  GENERAL  IMPRESSIONS  OF  EVENTS  DESCRIBED 

35F  SELECT  EXAMPLES  THAT  WILL  CLARIFY  MAJOR  ISSUES  PRESENTED  IN 
THE  REPORT 

35G  EXAMINE  OPPOSING  POINTS  OF  VIEW  IN  THE  REPORT 

35H  SUMMARIZE  THE  MAJOR  POINTS  DEVELOPED  IN  THE  REPORT 

351  JUSTIFY  AN  ACTION  TAKEN  AND  GIVE  REASONS  FOR  REJECTING 
ALTERNATIVES 

36  EDITING 

36A  SPELL  FREQUENTLY  USED  WORDS  CORRECTLY 

36B  SPELL  TASK-RELATED  WORDS  CORRECTLY 

36C  IDENTIFY  WORDS  THAT  NEED  TO  BE  CAPITALIZED 
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36D  CORRECT  ALL  MISSPELLED  WORDS  WITH  OR  WITHOUT  THE  USE  OF  A 
REFERENCE  SOURCE 

36E  APPLY  ALL  RULES  FOR  END  MARKS*  COMMAS*  AND  APOSTROPHES 

36F  APPLY  COMMON  RULES  OF  GRAMMAR 

36G  REWRITE  PARAS  *  STATE  MAIN  IDEA  IN  FIRST  SENTENCE, 

RESTRUCTURE  FOR  COHERENCE 

36H  APPRAISE  AN  ENTIRE  WRITTEN  COMMUNICATION  &  MAKE  ADJ.  TO 
IMPROVE  CLARITY 

37  TYPE 

37A  INDIVIDUAL  -  A  PERSON  WORKING  ON  A  TASK  AND  COMMMUNICATING 
WITH  ANOTHER 

37B  INSTRUCTION-  COMMUNICATION  BETWEEN  INSTR.  &  INDIVID.  OR 
GROUP  TO  PASS  FACTS 

37C  TUTOR-INTERACTION  BETWEEN  TWO  PERSONS  -  ONE  INSTRUCTING,  ONE 
DOING  THE  TASK 

37D  PEER  GROUP ( LESS  THAN  1 0 )-  ACTIVITY  WHERE  ONE  PERSON  ASSUMES 
LEADERSHIP  ROLE 

37E  INTERVIEW  -  COMMUNICATING  W/ANDTHER  TO  USE  THE  INFORMATION 
IN  A  TASK 


37F  BRIEFING  -  COMMUNICATING  FINAL  INSTRUCTIONS  OR  GIVING  AN 
ACCOUNT  IN  SUMMARY 

37G  COUNSEL  -  COMMUNICATING  TO  EXCHANGE  IDEAS  OR  OPINIONS  TO 
RECOMMEND*  ADVISE 

37H  COMMAND  -  COMMUNICATE  AN  ORDER  OR  ACTION  TO  BE  TAKEN 
38  CHARACTERISTICS 

38A  ENUNCIATE  CLEARLY*  USING  THE  PROPER  RATE  OF  SPEECH 

38B  USE  TECHNICAL  VOCABULARY  SUITABLE  TO  THE  TASK  AND  LEVEL  OF 
THE  PERSON 

38C  DETERMINE  THE  APPROPRIATE  AMOUNT  OF  INFORMATION  TO 
COMMUNICATE 
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38D  INTERPRET  FIGURATIVE  OR  IDIOMATIC  LANGUAGE  BY  REFERENCE  TO 
USE  IN  CONTEXT 

38E  FOLLOW  HIGHLY  DETAILED,  STEP-BY-STEP  DIRECTIONS 

38F  SOLICIT  FEEDBACK  TO  CONFIRM  THE  ACCURATE  RECEPTION  OF  THE 
COMMUNICATION 

38G  RECOGNIZE  WHEN  A  LOW-KEY,  INFORMAL  DIALOGUE  IS  SUITABLE 

38H  RECOGNIZE  WHEN  DIRECT  VERBAL  COMMANDS  ARE  NECESSARY 

381  RECOGNIZE  WHEN  A  SERIES  OF  VERBAL  INTERACTIONS  IS  REQUIRED 
TO  COORDINATE 

38 J  RECOGNIZE  WHEN  SITUATION  WILL  REQUIRE  A  STRUCTURED  METHOD  OF 
PRESENTATION 

39  BARRIERS 

39A  RECOGNIZE  NEED  FOR  CLEAR  CONCISE  DIRECTIONS*  AVOID  WORD¬ 
MEANING  DIFFERENCES 

39B  RECOGNIZE  PERSONALITY  FACTORS  &  INTER-PERSONAL  RELATIONSHIPS 
THAT  MAY  EXIST 

39C  RECOGNIZE  FEEDBACK  AIDS  EFFECTIVE  COMMUNICATION  &  INCREASES 
TASK  COMPETENCE 

AO  PRECAUTIONS 

A 0 A  USE  COMMON  KNOWLEDGE  TO  AVOID  HAZARDS  TO  PREVENT  INJURY  TO 
SELF  OR  EQUIP. 

AOB  APPLY  PREVENTIVE  MEASURES  PRIOR  TO  TASK*  MINIMIZE  SAFETY 
OR  SECURITY  PROBLEMS 

AOC  SELECT  AN  APPROPRIATE  COURSE  OF  ACTION  IN  THE  EVENT  OF  AN 
EMERGENCY 

At  RECOGNITION 

A1 A  IDENTIFY  SIMILARITIES  AND  DIFFERENCES  BETWEEN  AND  AMONG 
OBLECTS 

A1B  USE  BODY  LANGUAGE  (MOTIONS,  GESTURES,  POSTURES)  TO 
COMMUNICATE  OR  SIGNAL 
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4 1 C  DETERMINE  THE  PRESENCE  OF  A  DEFECT  OR  EXTENT  OF  DAMAGE 

41 D  MATCH  OBJECTS  BY  SIZE ,  SHAPE ,  COLOR,  AND  SIGNIFICANT  MARKING 


41E  CLASSIFY  OBJECTS  BY  SIZE,  SHAPE,  COLOR  AND  SIGNIFICANT 
MARKINGS 

4 1 F  DETERMINE  DIRECTION, DURATION,  AND  INTENSITY  OF  SOUNDS,  SIGHT 
SIGHTINGS, AND  SMELLS 

41 G  INFER  FROM  SIGHTS , SOUNDS , TOUCH , SMELLS , OR  TASTES  TO  DETERM 
COURSE  OF  ACTION 

4 1 H  INTERPRET  CODES  AND  SYMBOLS 
45  MOTOR  SKILL 

45A  MULTI-LIMB  COORDINATION:  COORDINATE  GROSS  MOVEMENTS 
REQUIRING  USE  OF  LARGE  SKELETAL  MUSCLES  FOR  STRENGTH 

45B  MANUAL .DESTERITY:  MAKE  SKILLFUL,  CONTROLLED  ARM/HAND 
JKMOVEHENTS  IN  MANIPULATING  LARGE  OBJECTS  UNDER 
SPEED  CONDITIONS 

45C  CONTROL  PRECISION:  HIGHLY  CONTROLLED,  PRECISE 

MUSCULAR  ADJUSTMENTS,  OPERATING  CONTROLS  BY  HAND, 

ARM  OR  FOOT  MOVEMENTS 

45D  RESPONSE  ORIENTATION:  SELECT  APPROPRIATE  RESPONSE,  EACH 
PATTERN  OF  SIGNALS  REQUIRES  A  DIFFERENT  CHOICE  OF 
CONTROLS  AND  DIRECTION  AND  MOVEMENT-SPEED  AND  ACCURACY 

45E  REACTION  TIME:  SPEED  WITH  WHCH  AN  INDIVIDUAL  IS  ABLE 
TO  RESPOND  TO  AN  AUDITORY  OR  VISUAL  STIMULUS 

45F  RATE  CONTROL:  MAKE  CONTINUOUS  ANITICPATORY  MOTOR 

ADJUSTMENTS  RELATIVE  TO  CHANGES  IN  SPEED  KOF  A  MOVING 
TARGET 

45G  FINGER  DEXTERITY:  SKILLFUL,  CONTROLLED  MANIPULATIONS 
OF  SMALL  OBJECTS,  PRIMARILY  FINGER  MOVEMENTS  OF  SMALL 

45H  ARM-HAND  STEADNESS:  MAKE  PRECISE  ARM-HAND  POSITIONING 
MOVEMENTS  WHERE  STRENGTH  AND  SPEED  ARE  MINIMIZED 

451  WRIST-FINGER  SPEED:  MAKE  RAID,  PR£CISE  TURNING, 
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TAPPING,  TRIPPING,  ALIGNING  MOTIONS  IN  RESPONSE  TO 
AUDITORY  OR  VISUAL  STIMULI 

46  OPERATIONS 

46A  OPERATE  EQUIPMENT  OR  BE  A  CREW  MEMBER  ENGAGED  IN 
MANUALLY  LIFTING  OR  MOVING  OBJECTS 

46B  USE  HAND  TOOLS  CONSIDERED  TO  BE  COMMON;  DETERMINE  SIZES  AND 

AND  SELECTION;  PLIERS,  SCREWDRIVERS,  HAND  DRILLS - SIMPLE 

MECHANICAL  OPERATIONS,  MAINLY  REMOVE  AND  REPLACE 

46C  INSTALL,  RELOCATE  AND  REMOVE 

46D  ADJUST,  REPAIR,  MAINTAIN  MECHANICAL,  ELECTRICAL, 

HYDRAULIC  SYSTEMS  OR  SUBSYSTEMS;  DISSEMBLE  AND  ASSEMBLE 
USING  HAND  TOOLS 

46E  TESTING  EQUIPMENT  USED  IN  THE  PRODUCTION,  TRANSMISSION 
DISTRIBUTION,  AND  UTILIZATION  OF  ENERGY  SOURCES 

46F  CURRING,  BENDING,  TRIMMING,  WELDING,  RIVETING  ETC. 

TO  FORCE,  SHAP  OR  FABRICATE  METALS 

46G  OPERATOR  CHECKS  AND  SERVICES  IN  RESPONSE  TO  A  VARIETY 
OF  EXTERNAL  SIGHTS  AND  SOUNDS 

46H  POSSESS  THE  PHYSICAL  ABILITY  TO  PERFORM  IN  COMBAT 
OR  POLICE  ACTIONS 

461  CLERICAL  DUTIES,  MAINTAINING  RECORDS,  OPERATING 
OFFICE  MACHINES,  PROCESSING  DATA  USING  ELECTRONIC 
MACHINES 

46 J  PERFORM  ACTS  REQUIRING  SKILL  AND  CARE  OF  INJURED  OR 
INFIRMED  PERSONS 


47  EQUIPMENT  USE 

48  OTHER 
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Appendix  E 
76C  (PLL)  Test  Plan 
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Test  Plan  for  76C  AIT  Dut^  Position  Annexes 

This  test  plan  specifies  the  kind  and  number  of  items  to  be 
used  in  developing  performance  tests  for  the  four  duty  position 
annexes  taught  in  the  76C  AIT  course:  PLL  (manual  and 
automated),  TAMMS,  shop  stock  (manual  and  automated),  and  shop 
clerk.  Alternate  forms  of  the  end  of  the  annex  test  should  be 
constructed  within  the  framework  of  the  plan.  Each  of  the  tests 
must  cover  the  critical  tasks  for  that  annex. 

The  test  plans  are  designed  as  part  of  a  CMI  system  in  which 
scores  are  computed  and  analyzed  by  the  hardware/software  system 
in  operation.  For  on-line  tests,  trainees  will  answer  test  items 
on  the  terminal  screen  and  their  test  scores  will  be  computed 
directly  (cf.  prototype  CMI  system  [Interamerica,  1986]).  For 
off-line  tests,  the  answers  to  tests  are  typed  individually  into 
the  CMI  system  using  appropriate  test  and  student  identification, 
which  then  computes  the  score. 

Each  test  item  is  assigned  a  taxonomic  code  based  on  the 
type  of  information  or  knowledge  needed  to  answer  it  correctly 
(see  Appendix  D  for  the  diagrams  of  the  taxonomic  plans  for  each 
of  the  four  duty  positions  in  the  76C  MOS) .  The  software  program 
tallies  the  correct  and  incorrect  responses  for  each  student  and 
provides  figures  on  the  proportion  choosing  each  item  alternative 
in  multiple  choice  questions.  In  addition,  the  items  missed  on 
each  taxonomic  category  cell  are  summed  and  provided  as  output 
tables.  From  these  tables,  instructors  can  quickly  determine  the 
particular  functional  areas  which  have  not  been  mastered  by  the 
trainee.  Summations  should  also  be  provided  by  row  and  column  of 
the  taxonomic  matrix.  For  example,  if  a  trainee  misses  a  PLL 
item  with  code  6C  (Request  for  issue,  non-NSN  repair  part  -  form 
fill  out)  and  two  items  with  code  4C  (PLL  maintenance  -  form 
fill  out)  there  would  be  one  error  in  column  4,  one  error  in 
column  6,  and  3  errors  in  row  C.  Thus,  the  trainee  would  be 
exhibiting  particular  problems  with  form  fill  out  operations. 

PLL  Annex  (Manual) 

The  end  of  annex  test  for  the  PLL  (Manual)  is  composed  of  7 
form  fill  out  items  and  35  multiple-choice  items.  The  form  fill 
out  items  will  be  worth  a  total  of  65  points.  Thirty-five 
multiple-choice  items  will  be  worth  1  point  each  for  35  points 
total.  Thus  the  test  as  a  whole  will  be  worth  100  points. 

Form  Fill  Out  Items 

The  form  fill  out  items  will  contain  versions  of  the  main 
forms  used  in  the  course  of  the  PLL  clerk's  duties.  The  set  of 
forms  to  be  used  are:  the  DA  Form  2765-1,  DD  Form  1348-6,  DA  Form 
2402,  DA  Form  3318,  DA  Form  2063-R,  and  DA  Form  2064.  The  six 
items  should  constitute  a  representative  set  of  the  functional 
activities  that  the  PLL  clerk  must  perform  in  a  typical  day.  The 
following  critical  tasks  will  be  covered  in  these  items  as  of 
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February  1986  (cf .  Appendix  A) :  a)  process  a  request  for  a 
prescribed  load  list  (PLL)  repair  part,  2)  process  a  request  for 
nonstockage  list  repair  part,  3)  process  a  request  for  non¬ 
national  stock  number  repair  part,  4)  process  request  for  repair 
part  designated  as  RX  (repairable  exchange) ,  5)  turn-in  repair 
parts,  6)  conduct  review  and  inventory  of  PLL  records,  7)  process 
supply  and  shipment  status  list  or  cards,  8)  initiate  follow  up 
or  document  modification  action.  Optionally,  the  following  tasks 
may  be  covered  in  addition:  1)  prepare  and  maintain  nonstockage 
list  records,  2)  maintain  a  PLL,  3)  initiate  cancellation  action, 
and  4)  perform  reconciliation  of  due-in. 

The  purpose  of  the  form  fill  out  items  is  the  measurement  of 
the  trainees  ability  to  correctly  fill  out  the  major  forms 
currently  being  used  in  the  course  of  the  work  performed  by  the 
PLL  (manual)  clerk.  The  test  taker  will  be  provided  with  a  set 
of  information  which  would  be  generally  available  to  him  in  the 
job  environment  in  order  to  fill  out  the  forms  correctly, 
including  relevant  technical  manuals,  DA  pamphlets  and  supply 
unit  codes.  Specifically,  these  materials  will  be  provided  in 
advance  to  the  test  taker:  1)  technical  manuals  (excerpts) 
containing  the  NSN,  figure  number,  item  number  for  non-stock 
supply  parts  named,  2)  parts  on  a  repairable  exchange  (RX)  list, 
3)  DA  Pam  710-2-1  including  appendices,  4)  a  unit  PLL  containing 
NSN,  item  description  unit  of  issue,  and  quantity  for  stock 
parts,  and  5)  general  unit  information  (e.g.,  DIC,  FAD,  DODAAC, 
SSA,  location,  CO) . 

Each  of  the  seven  form  fill  out  items  should  be  independent 
of  the  responses  to  the  other  items  although  they  may  deal  with 
the  same  class  of  parts.  The  items  should  function  as  an 
integrated  series  of  activities  comprising  "a  day  in  the  life" 
of  a  PLL  clerk. 

Item  Format 

Item  A.  This  item  tests  the  ability  to  complete  a  request  for 
issue  for  PLL  parts  ar.d  nonstockage  parts  possessing  an  NSN  and 
the  requisite  posting  of  the  transactions  to  the  supply  records. 
The  following  information  is  provided  to  the  test  taker  in 
addition  to  the  general  information: 

a)  the  name  of  5  parts  to  be  issued  or  ordered  there 
at  least  one  and  not  more  than  2  parts  will  be  non 
stockage  NSN  parts, 

b)  5  unfilled  DA  Form  2765-1.  Preprinted  versions  may 

be  used  for  those  parts  which  are  on  the  PLL, 

c)  narrative  information  sufficient  to  derive  a  UND 
and  the  end  item, 

d)  one  filled  in  DA  Form  2064, 

e)  one  DA  Form  3318  with  at  least  one  previous  demand 
and  request  for  each  of  the  PLL  parts, 

f)  one  unfilled  DA  Form  3318  for  each  of  the  non- 
stocked  parts. 
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A  scenario  which  provides  the  basis  for  ordering  the  parts  will 
be  provided  which  will  be  representative  of  a  typical  job 
situation.  The  test  taker  will  be  asked  to  fill  out  the 
following  information  on  the  preprinted  DA  Form  2765:  a)  quantity 
requested  with  unit  of  issue,  b)  Julian  date/serial  number,  c) 
EIC,  d)  PD,  e)  NMCS  or  ANMCS ,  f)  item  information  (NSN,  name) . 
With  respect  to  the  manual  DA  Form  2765s,  the  test  taker  will 
fill  out  the  information  listed  above  plus  a)  the  DIC,  b) , 

DODAAC ,  and  c)  demand  code. 

In  the  second  part,  after  all  the  DA  Form  2765s  have  been 
filled  in,  the  test  taker  will  post  the  supply  actions  to  the 
document  register  (DA  Form  2064) .  The  test-taker  will  enter:  a) 
the  Julian  date,  b)  serial  number,  c)  the  last  three  digits  of 
the  SSA  DODAAC,  d)  the  NSN,  e)  item  name,  f)  PD,  g)  quantity 
requested,  and  h)  Julian  follow  up  due  date. 

In  the  third  part,  the  test  taker  will  take  the  appropriate 
action  to  post  the  requests  to  the  Record  of  Demands  DA  Form 
3318.  For  PLL  items,  the  respective  DA  Form  3318  will  have  the 
title  insert  filled  in;  in  addition,  at  least  one  demand  and  one 
request  will  be  listed.  For  the  non  PLL  items  the  respective  DA 
Form  3318s  will  have  no  information  in  the  title  insert. 

However,  the  test  taker  will  supply  the:  a)  NSN,  b)  document 
number,  c)  quantity  requested,  d)  quantity  due  in,  e)  end  item, 

:)  item  description. 

Item  Scoring  (A):  This  item  is  worth  a  total  of  5  points.  One 
or  more  errors  in  the  following  parts  of  the  DA  Form  2765  will  be 
penalized  by  the  full  5  points:  a)  quantity  requested,  b)  EIC, 

c)  PD,  d)  item  information,  e)  unit  issue.  The  full  five  points 

are  deducted  since  these  errors  could  lead  to  inaccurate 
ordering  of  the  items  and  are  indicative  of  the  test  taker's 
technical  knowledge  for  PLL  manual  procedures.  Multiple  errors 
are  not  additionally  penalized.  One  or  more  errors  in  the 
following  parts  of  the  DA  Form  2765  are  penalized  a  total  of  one 
point:  a)  Julian  date  and  serial  number,  b)  DIC,  c)  DODAAC,  and 

d)  demand  code. 

One  or  more  errors  in  the  DA  Form  2064  are  penalized  a  total  of 
one  point.  One  or  more  errors  in  the  DA  Form  3318s  are  penalized 
a  total  of  one  point. 

Example:  A  test  taker  makes  an  error  on  the  Julian  date  in  the  DA 
Form  2765,  two  errors  on  the  DA  Form  2064,  and  two  errors  on  the 
DA  Form  3318s.  A  total  of  3  points  are  deducted;  one  point  for 
missing  the  Julian  date,  one  point  for  the  one  (or  more)  errors 
on  the  DA  Form  2064,  and  one  point  for  the  one  (or  more)  errors 
on  the  DA  Form  3318. 
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Item  B.  This  item  tests  the  capacity  to  complete  a  turn-in  of 
parts.  The  following  information  is  provided  to  the  test  taker  in 
addition  to  the  general  information: 

a)  the  names  of  the  parts  to  be  turned  in, 

b)  two  unfilled  DA  Form  2765-ls. 

A  scenario  which  provides  the  basis  for  turning  in  the  parts  will 
be  provided  which  will  be  representative  of  a  typical  job 
situation.  The  test  taker  will  be  asked  to  fill  out  the 
following  information  on  the  DA  Form  2765-1:  a)  quantity  being 
turned  in,  b)  NSN  (if  applicable) ,  c)  item  description,  d) 
condition,  e)  DODAAC ,  f)  Julian  date  and  serial  number,  g)  unit 
making  turn  in,  and  h)  date  turn-in  completed. 

Item  Scoring.  This  item  is  worth  a  total  of  5  points.  One  or 
more  errors  in  the  following  parts  of  the  DA  Form  2765-ls  will  be 
penalized  by  deducting  5  points:  a)  condition,  b)  NSN  (if 
applicable)  one  or  more  errors  in  the  following  parts  of  the  DA 
Form  2765-1  will  be  penalized  by  deducting  one  point,  a) 
quantity,  b)  item,  c)  DODAAC,  d)  Julian  date  and  serial  number, 
e)  unit  making  turn  in,  and  f)  date  turn  in  completed. 

Item  C.  This  item  tests  the  capacity  to  order  non-NSN  parts. 

The  following  information  is  provided  to  the  test  taker  in 
addition  to  the  general  information: 

a)  the  names  of  2  non-NSN  parts  to  be  ordered, 

b)  the  name  of  1  NSN  part  which  does  not  need  to  be 
ordered, 

c)  narrative  information  sufficient  to  derive  a  UND 

d)  two  unfilled  DD  Form  1348-6s  for  the  non-NSN  parts 
to  be  ordered, 

e)  an  excerpt  from  the  MCR  to  determine  if  the  parts 
do  have  NSNs. 

A  scenario  which  provides  the  basis  for  ordering  the  parts  will 
be  provided  which  will  be  representative  of  a  typical  job 
situation.  The  test  taker  will  be  asked  to  determine  which  of 
the  parts  do  not  have  an  NSN  and  then  order  those  parts.  The 
following  information  will  be  filled  out  on  the  DD  Form  1348-6: 
a)  part  number,  b)  FSCM,  c)  item  description,  d)  quantity,  e) 
unit  of  issue,  f)  PD,  8)  Julian  date  and  serial  number,  h) 

DODAAC . 

Item  Scoring.  This  item  is  worth  a  total  of  5  points.  One  or 
more  errors  in  the  following  parts  of  the  DD  Form  1348-6s  will 
be  penalized  by  deducting  5  points:  a)  part  number,  b)  FSCM,  c) 
unit  issue,  d)  PD,  and  e)  quantity  requested.  One  or  more  errors 
in  the  following  parts  of  the  DD  Form  1348-6  will  be  penalized 
by  deducting  one  point:  a)  Julian  date/serial  number,  b)  DODAAC, 
and  c)  item  description. 
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Item  D.  This  item  tests  the  capacity  to  execute  a  RX  request. 
The  following  information  is  provided  to  the  test  taker  in 
addition  to  the  general  information: 

a)  the  name  of  the  RX  parts  to  be  ordered, 

b)  narrative  information  sufficient  to  derive  a  UND 
and  determine  the  PD  code, 

c)  two  unfilled  RX  slips  (DA  Form  2765-1) , 

d)  two  DA  Form  3318s  with  at  least  one  previous  demand 
and  request. 

A  scenario  which  provides  the  basis  for  exchanging  RX  parts  will 
be  provided  which  will  be  represented  of  a  typical  job  situation. 
The  test  taker  will  be  asked  to  fill  out  the  following 
information  on  the  RX  slip  (DA  Form  2765-1) :  a)  NSN,  b)  PD,  c) 
deficiency,  d)  DODAAC ,  e)  serial  #,  f)  NMCS,  g)  noun,  h)  end 
item,  and  i)  model. 

Item  Scoring.  This  item  is  worth  a  total  of  5  points.  One  or 
more  errors  in  the  following  parts  of  the  DA  Form  2765  will  be 
penalized  by  the  full  5  points,  a)  NSN,  b)  PD,  c)  serial  #,  d) 
noun,  e)  end  item,  and  i)  model. One  or  more  errors  in  filling  out 
the  DA  Form  3318s  will  be  penalized  one  point. 

Item  E.  This  item  tests  the  capacity  to  review  demands  and  make 
the  required  changes  to  the  unit's  PLL.  The  following 
information  is  provided  to  the  test  taker  in  addition  to  the 
general  information: 

a)  four  filled  out  DA  Form  3318s  for  two  PLL  and  two 
non-PLL  items  which  require  review, 

b)  PLL  stockage  level  task, 

c)  one  filled  out  DA  Form  2063-R  (consolidated) , 

d)  one  blank  DA  Form  2063-R. 

A  scenario  which  provides  the  basis  for  a  review  of  demands  and 
consequent  changes  to  the  PLL  will  be  provided  to  the  test  taker. 
The  test  taker  will  have  to  perform  the  following  potential 
actions:  a)  determine  whether  a  review  is  needed  for  the  item, 

b)  determine  any  changes  in  authorized  stock  level,  c)  determine 
any  changes  to  PLL,  d)  make  appropriate  changes  on  DA  Form  3318s 
in  the  event  of  stock  level  increases  or  decreases  (e.g.  post 
additional  requests,  cancel  excess  due-in.  The  test  taker  will 
fill  out  a  revised  DA  Form  2063-R  (consolidated)  performing  the 
following  actions:  a)  arranging  items  in  NUN  sequence,  b) 
providing  item  description,  c)  unit  of  issue,  d)  authorized 
quantity,  and  e)  remarks  (e.g.  added  changes  or  deleted) .  In  the 
second  part  the  test  taker  will  make  appropriate  notations  on  the 
DA  Form  3318s  such  as  a)  adjust  due  in  levels,  b)  change 
authorized  stockage  levels,  and  c)  post  additional  requests. 
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Item  Scoring.  This  item  is  worth  a  total  of  5  points.  One  or 
more  errors  in  the  following  parts  of  the  DA  Form  2063-R  will  be 
penalized  the  full  5  points:  a)  incorrect  listing  of  items,  b) 
incorrect  stockage  levels  One  or  more  errors  in  the  following 
parts  of  the  2063-R  will  be  penalized  by  one  point:  a) 
incorrect  NUN  sequence,  b)  unit  of  issue,  and  c)  remarks. 

One  or  more  errors  in  the  following  parts  of  the  DA  Form 
3318  will  be  penalized  by  2  points:  a)  adjusting  due  in  levels 
and  b)  posting  additional  request.  One  or  more  errors  in  the 
following  parts  of  the  DA  Form  3318  will  be  penalized  by  one 
point:  a)  changes  in  authorized  stockage  code. 

Item  F.  This  item  tests  the  ability  to  post  status  and  receive 
repair  parts.  The  following  information  is  provided  to  the  test 
taker  in  addition  to  the  general  information: 

a)  3  receipts,  two  of  which  are  copies  of  preprinted 
DA  Form  2765s  from  the  original  request  and  one  of 
which  is  a  filled  in  DA  Form  1348-1, 

b)  2  supply  status  cards  on  preprinted  DA  Form  2765s, 

c)  one  DA  Form  2064, 

d)  3  DA  Form  3318s  one  for  each  part  (receipt) , 

e)  Appendix  C:  (Status  Rejection  codes) . 

A  scenario  which  provides  the  basis  for  receiving  parts 
(receipts)  and  supply  status  cards  will  be  provided  which  will  be 
representative  of  a  typical  situation.  The  test  taker  will  be 
asked  to  fill  out  the  following  information  on  the  DA  Form  2064 
after  matching  the  document  number,  NSN,  quantities  and 
nomenclature  of  the  receipt  with  the  line  item:  a)  post  the 
quantities  received  from  the  receipt  document,  b)  erase  or  change 
the  quantity  due-in,  and  c)  enter  date  received. 

If  the  item  is  a  stockage  item,  post  the  following 
information  to  the  appropriate  DA  Form  3318  a)  change  the 
quantity  due-in,  b)  if  ordered  for  a  vehicle  change  the  quantity 
due  in  and  c)  if  the  item  is  for  stockage,  adjust  balance  on 
hand. 


If  it  is  a  partial  quantity  receipt  test  taker  must  post  the 
actual  quantity  received  to  the  DA  Form  2064,  b)  new  quantity 
due-in,  and  c)  date  available  with  appropriate  status  code. 

Then,  if  appropriate,  he  should  post  to  the  DA  Form  3318  the  new 
quantity  due-in. 

In  the  second  part  the  test  taker  should  post  supply  status 
information  to  the  DA  Form  2065  (i.e.  cancellation  of  PLL  items) 
and  where  appropriate  to  the  DA  Form  3318.  The  information  posted 
will  be  the  a)  supply  status,  b)  estimated  delivery  date,  c) 
cancellation  (if  any)  in  remarks  section.  The  cancellation  will 
be  posted  to  the  DA  Form  3318  as  well  by  changing  the  quantity 
requested  and  due-in. 
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Item  Scoring.  This  item  is  worth  a  total  of  five  points.  One  or 
more  errors  in  the  following  parts  of  the  DA  Form  2064  will  be 
penalized  2  points:  a)  changes  in  quantity  due-in,  b)  supply 
statutes  (EDD) .  One  or  more  errors  in  the  following  parts  of  the 
DA  Form  2064  will  be  penalized  one  point:  a)  quantities  received 
and  b)  date  received.  One  or  more  errors  in  the  following  parts 
of  the  DA  Form  3318  will  be  penalized  2  points:  a)  change 
quantity  due-in  if  ordered  for  vehicle,  b)  changes  in  the  balance 
on  hand,  and  c)  cancellation  postings.  One  or  more  errors  in  the 
following  parts  of  the  DA  Form  3318  will  be  penalized  on  point: 
quantities  received. 

Item  G.  This  item  tests  the  knowledge  of  conducting  and  posting 
follow-ups  and  one  of  the  following  activities:  modifications  or 
cancellations.  The  following  will  be  provided  the  test  taker  in 
addition  to  the  general  information: 

a)  one  filled  out  DA  Form  2064, 

b)  two  blank  DA  Form  2765s  (tor  follow-ups) , 

c)  two  status  card  DA  Form  2765s  (for  cancellations) , 

d)  two  blank  DA  Form  2765s  (for  document  modifier 
requests) , 

e)  reconciliation  list  (for  reconciliations  and 
follow-ups) . 

Scenarios  which  provide  the  basis  for  two  of  the  four  activities 
listed  above  will  be  provided  to  the  test  taker  which  will  be 
representative  of  a  typical  job  situation. 

For  follow  ups  the  test  taker  will  be  asked  to  determine 
from  the  DA  Form  2064  (or  reconciliation  list)  which  items  (no 
more  than  2)  require  follow  ups.  For  those  items  which  require 
follow  ups  the  test  taker  will  fill  in  the  following  information 
on  the  DA  Form  2765s:  a)  DODAAC ,  b)  Julian  date  and  serial 
number,  c)  NSN,  d)  item  description,  e)  priority,  f)  quantity 
ordered  from  original  request,  g)  NMCS/ANMCS  identifier,  h) 
document  identifier  code,  i)  recurring  or  nonrecurring,  j)  unit 
of  issue,  and  k)  EIC.  The  information  will  be  obtained  from  the 
DA  Form  2064  and  the  appropriate  DA  Form  3318. 

In  the  second  stage,  the  test  taker  will  post  the  follow-ups 
to  the  DA  Form  2064  by  erasing  the  date  the  follow-up  is  due  and 
status  in  column  L  and  entering  AF1  and  the  date  of  the 
transaction  in  column  N. 

For  cancellation,  the  test  taker  will  be  asked  to  cancel  one 
or  two  requests  using  the  status  cards  (DA  Form  2765)  provided. 
The  test  taker  will  fill  in  the  a)  DIC,  b)  quantity  to  be 
cancelled,  and  c)  routing  identifier. 

In  the  second  stage,  the  cancellations  are  posted  to  the  DA 
Form  2064  by  entering  the  code  AC1  and  the  Julian  date  in  column 
N. 
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For  document  modifier  requests  the  test  taker  will  be  asked 
to  change  the  priority  and/or  the  require  delivery  of  one  or  two 
requests  using  the  status  cards  (DA  Form  2765)  provided.  The 
test  taker  will  fill  in  the  a)  DIC,  b)  routing  identifier  code, 
c)  priority) ,  d)  NMCS/ANMCS  if  required  and  e)  the  required 
delivery  date. 

In  the  second  stage,  the  modification  will  be  posted  to  the 
DA  Form  2064.  The  test  taker  will  erase  the  entry  in  column  L 
and  enter  the  DIC  and  Julian  date  in  column  N.  If  appropriate, 
the  priority  (required  delivery)  should  be  crossed  out  and  the 
new  one  entered  next  to  it.  If  necessary,  changes  to  the 
document  number  must  be  made  as  well. 

Item  Scoring.  This  item  is  worth  a  total  of  5  points.  For 
follow-ups,  one  or  more  errors  in  the  DA  Form  2765  will  be 
penalized  2  points.  One  or  more  errors  in  the  DA  Form  2064  will 
be  penalized  one  point.  For  cancellations  and  document  modifier 
requests  one  or  more  errors  in  the  DA  Form  2765  status  cards  will 
be  penalized  one  point.  One  or  more  errors  in  the  DA  Form  2064 
will  be  penalized  one  point. 

Multiple-Choice  Items 

The  end  of  annex  PLL  test  will  contain  multiple-choice  items 
in  addition  to  the  fill-in  the  form  items.  The  36  items  will 
cover  8  topic  areas:  (1)  requests  for  issue  NSN,  (2)  turn-ins, 

(3)  RX  and  QSS,  (4)  PLL  maintenance,  (5)  supply  status  and 
receipts,  (6)  requests  for  issue  (non-NSN) ,  (7)  follow-ups, 

modifications,  cancellations,  reconciliations,  and  (8)  general 
information.  Thirty  of  the  items  will  be  drawn  from  the  first 
seven  categories  with  each  category  being  represented  by  no  less 
that  2  and  no  more  than  8  items.  Each  of  these  30  items  will  be 
worth  2  points.  The  general  information  category  will  contain  5 
items.  Each  of  these  items  will  be  worth  one  point. 

The  items  with  two  points  will  cover  KSAs  concerning  the 
functional  activities  of  the  PLL  clerk.  These  items  will  be 
constructed  with  a  stem  and  four  alternatives:  A,  B,  C,  D.  The 
items  will  deal  with  5  types  of  queries:  1)  form  functions,  2) 
definitions,  3)  form  fill  out,  4)  form  related  information,  and 
5)  postings.  Form  functions  involve  the  uses  to  which 
particular  forms  are  put  and  the  role  they  play  in  the  activities 
of  the  PLL  clerk.  Definitions  involve  the  meaning  of  key  terms 
used  in  the  different  functional  activities.  Form  fill  out 
involves  the  way  in  which  the  different  forms  need  to  be  filled 
out  in  order  to  fulfill  their  purpose  and  the  kind  of 
information  which  needs  to  be  entered.  Form  related  information 
involves  additional  knowledge  (e.g.  PDs,  Julian  dates)  needed  to 
complete  the  required  forms.  Posting  involves  the  information 
from  one  form  which  needs  to  be  transferred  and  recorded  on 
another  form  for  record  keeping  purposes. 
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The  items  worth  one  point  will  cover  the  general  nature  and 
function  of  the  PLL  duty  position  as  it  relates  to  other  76C  duty 
positions  and  the  supply  system  in  general.  These  items  will  be 
constructed  with  a  stem  and  four  alternatives:  A,  B,  C,  D. 
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Appendix  F 

Job  History  Frequency  Counts 


PERSONNEL  BY  SITE 


[~~|  Ft.  Riley 
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CURRENT  DUTY 
POSITION 


^  SHOP  Clerk 


TAMMS  Clerk 
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TIME  IN  CURRENT 
POSITION 


EB  1-2  years 
Iff  Less  than  1  year 


PRIOR  DUTY 
POSITION 


TAMMS  Clerk 
PLL  Clerk 
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□ 
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HIGH 
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|  |  FREQUENT 


SUPERVISOR  RATINGS 
by  SITE 


n  „ 


Appendix  G 

Supervisor  Ratings  Frequency  Counts 
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In  this  appendix,  the  results  for  the  individual  scales  of 
the  supervisor  rating  form  are  presented  in  the  accompanying 
graphs. 

The  findings  for  Scale  A  have  already  been  discussed. 
Supervisors  rated  57%  of  the  soldiers  as  putting  in  extra  effort 
(Scale  B)  when  completing  important  or  time-critical  assignments 
in  order  to  overcome  obstacles.  While  17%  did  not  put  in  any 
extra  effort  or  even  gave  up  in  such  situations,  almost  26%  went 
above  and  beyond  what  was  asked  of  them  and  often  volunteered  to 
work  extra  hours  according  to  supervisors.  With  Scale  C,  the 
supervisors  believed  that  more  than  half  (52%)  of  the  soldiers 
almost  always  follow  Army  rules  and  regulations.  An  additional 
33%  of  the  soldiers  carefully  followed  the  rules,  while  15% 
often  failed  to  follow  rules  or  at  times  showed  signs  of 
disrespect. 

On  Scale  D,  48%  of  the  soldiers  were  rated  as  responsible 
and  truthful  in  dealing  with  mistakes  and/or  personal  matters. 

In  addition,  38%  take  extra  steps  to  ensure  that  others  aren't 
blamed  for  their  mistakes  and  are  honest  even  when  it  is  not  in 
their  best  interest.  However,  14%  were  seen  as  avoiding 
responsibility  and  assignments  or  not  displaying  sufficient 
integrity. 

In  the  area  of  leadership  and  ability  to  guide  others 
(Scale  E) ,  the  supervisor  saw  55%  of  the  soldiers  fully  capable 
in  the  context  of  76C  requirements.  Another  20%  of  the  group 
were  seen  as  able  to  take  charge  effectively  whenever  necessary 
(e.g.,  NCO  absence). 

While  in  the  Army,  soldiers  are  assigned  equipment.  The 
supervisors  rated  66%  of  the  76Cs  as  keeping  equipment  in  good 
condition  by  performing  routine  checks  and  preventive 
maintenance  while  19%  kept  their  equipmnt  in  'ready  for 
inspection'  condition  (  Scale  F) .  Only  15%  of  the  soldiers  kept 
equipment  in  poor  condition  and  rarely  performed  checks. 

While  nearly  one-half  (47%)  of  the  soldiers  were  seen  as 
being  neatly  dressed  and  maintaining  military  standards  of 
personal  cleanliness  and  grooming,  an  additional  43%  exceeded 
even  this  standard  (Scale  G) .  The  remaining  10%  were  rated  as 
sloppily  dressed  and  unable  to  meet  military  standards. 

The  majority  of  the  soldiers  (55%)  were  rated  in  good 
physical  condition  (Scale  H) ,  with  an  additional  34%  in 
excellent  condition  exceeding  Army  standards.  Only  10%  of  the 
soldiers  rated  were  seen  as  being  in  poor  condition. 

The  supervisor  in  Scale  I  rated  how  effective  soldiers  were 
in  developing  their  own  job  and  soldiering  skills.  About  83% 
of  the  soldiers  practiced,  studied  manuals,  or  participated  in 
courses  and  training  to  improve  his/her  skills  to  varying 
degrees.  The  remaining  17%  did  not  attempt  to  improve  their  job 
skills. 

The  supervisors  did  not  feel  that  many  soldiers  had  a 
problem  with  self  control.  They  perceived  that  only  9%  would 
lose  their  temper  easily.  The  other  91%  kept  their  temper  in 
most  situations.  About  one-half  of  this  latter  group  always  kept 
a  cool  head  and  avoided  aggressive  acts. 
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Supervisors  were  then  asked  to  rate  the  degree  of  confidence 
in  their  own  ratings  of  their  subordinates.  Two-thirds  of  the 
supervisors  were  highly  confident  that  their  ratings  of 
subordinate  effectiveness  were  accurate.  However,  the  remainder 
indicated  some  reservations  about  the  accuracy  of  their 
perceptions  of  subordinate  performance. 
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